Fails to load .ruby-lsp/Gemfile when Bundler version mismatches

[akihikodaki]

Description

Reproduction steps

1. Create Gemfile.lock with some non-default version of Bundler
2. Run ruby-lsp

Code snippet or error message

$ ruby-lsp
Ruby LSP> Skipping custom bundle setup since /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/Gemfile.lock already exists and is up to date
Ruby LSP> Running bundle install for the custom bundle. This may take a while...
Ruby LSP> Command: (bundle check || bundle install) 1>&2
The Gemfile's dependencies are satisfied
bundler: failed to load command: ruby-lsp (/home/me/.local/share/gem/ruby/bin/ruby-lsp)
/var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/rubygems_integration.rb:308:in `block in replace_bin_path': can't find executable ruby-lsp for gem ruby-lsp. ruby-lsp is not currently included in the bundle, perhaps you meant to add it to your Gemfile? (Gem::Exception)
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/rubygems_integration.rb:336:in `block in replace_bin_path'
        from /home/me/.local/share/gem/ruby/bin/ruby-lsp:25:in `<top (required)>'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/cli/exec.rb:58:in `load'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/cli/exec.rb:58:in `kernel_load'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/cli/exec.rb:23:in `run'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/cli.rb:486:in `exec'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/cli.rb:31:in `dispatch'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/cli.rb:25:in `start'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/exe/bundle:48:in `block in <top (required)>'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/lib/bundler/friendly_errors.rb:120:in `with_friendly_errors'
        from /var/home/me/HCK-CI/repos/AutoHCK.git/.ruby-lsp/vendor/bundle/ruby/3.3.0/gems/bundler-2.3.26/exe/bundle:36:in `<top (required)>'
        from /usr/bin/bundle:25:in `load'
        from /usr/bin/bundle:25:in `<main>'

Please note that while gems are installed in an unexpected place, it has nothing to do with this issue; the problem was reported with #2531.

It seems to be failing to load .ruby-lsp/Gemfile due to the following procedure:

1. exe/ruby-lsp requires lib/ruby_lsp/setup_bundler.rb.
2. lib/ruby_lsp/setup_bundler.rb requires bundler.
3. Bundler runs Bundler::EnvironmentPreserver#replace_with_backup, which sets BUNDLER_ORIG_BUNDLE_GEMFILE to BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL.
4. exe/ruby-lsp executes bundle exec ruby-lsp with BUNDLE_GEMFILE set.
5. Bundler detects the version mismatch and calls Bundler::SelfManager#restart_with.
6. Bundler::SelfManager#restart_with calls Bundler.with_original_env, which deletes BUNDLE_GEMFILE due to BUNDLER_ORIG_BUNDLE_GEMFILE.
7. Bundler::SelfManager#restart_with starts the bundled version of Bundler.
8. The bundled version of Bundler fails due to the missing BUNDLE_GEMFILE.

[vinistock]

Thank you for the detailed bug report!

I couldn't reproduce the bug, but I believe #2658 will fix it. Are you able to reproduce this issue reliably? If so, could you please test if the PR fixes the issue for you?

[akihikodaki]

I confirmed #2658 fixes the issue. Thanks for your work!