Test results are quite slow due to running each spec individually

[myronmarston]

First off, great work on this extension. It was really nice to be able to install it and have it "just work".

There is one thing that's not working great, though: the test results are very slow, and it appears that it's running each example individually (requiring a new ruby process to run and load rspec and my project code for each test!).

For example, here's a screenshot of the output I'm seeing:

Notice that each spec takes roughly 800 ms. When I run the entire spec file by hand in my terminal, here's the results:

Finished in 0.01117 seconds (files took 0.07163 seconds to load)
48 examples, 0 failures

Randomized with seed 11405

bundle exec rspec   0.26s user 0.14s system 51% cpu 0.766 total

I'm also seeing output in "Test Results" that shows that it's filtering to each example individually:

                                                                                     HashUtil
                                                                                                                      .stringify_keys
                                                                                                                                         replaces symbol keys with string keys

                     Top 1 slowest examples (0.00096 seconds, 63.4% of total time):
                                                                                     HashUtil.stringify_keys replaces symbol keys with string keys
                    0.00096 seconds ./path/to/spec.rb:11

                                                                                                               Finished in 0.00151 seconds (files took 0.07709 seconds to load)
                      1 example, 0 failures

                                           Randomized with seed 58416

                                                                     Run options: include {:locations=>{"./path/to/spec.rb"=>[7]}}

                                Randomized with seed 59884

                                                          HashUtil
                                                                                           .stringify_keys
                                                                                                              leaves a hash with string keys unchanged

                                                                                                                                                      Top 1 slowest examples (0.00089 seconds, 64.1% of total time):
                                                             HashUtil.stringify_keys leaves a hash with string keys unchanged
                                                                                                                                                        0.00089 seconds ./path/to/spec.rb:7

                                                                                         Finished in 0.00138 seconds (files took 0.07625 seconds to load)
                                                                                                                                                        1 example, 0 failures

                    Randomized with seed 59884

(And yes, the output I'm seeing there is formatted really weirdly. Not sure why, but I left it in so you could see it. Not a big deal though.)

Could ruby-lsp-rspec run the full set of tests it intends to run as a single rspec command instead of running rspec separately for each test?

[st0012]

I think the test result formatting issue is not RSpec or addon-specific, because I also see it in projects that don't use RSpec and this addon. Would you mind opening an issue in the extension repo?

Wrt the main issue, I don't think there's much this project can do either. If you click the Run In Terminal code lens on a test group, you'd notice that the command points to the group and tests are run in the same process as we'd expect. The same command is pushed to the extension and thus VS Code as well. But it looks like either VS Code doesn't use test group commands directly, or the vscode-ruby-lsp is not communicating with it properly. So I think the discussion should happen in vscode-ruby-lsp instead.

[cschneid]

Dropping some direct links here to follow along. Looks like at least some work has been done on running minitest tests together, which I guess would help this repo eventually as well.

Shopify/ruby-lsp#2402 and Shopify/ruby-lsp#2230