Towards no longer needing nvim/nvim-lspconfig

[ckipp01]

Question

As a follow-up to https://github.com/ckipp01/nvim-metals/pull/12#issuecomment-628661919, I think the setup needs to be addressed sooner rather than later.

Currently everything needs to be defined. For example, if you want to enable metals/status, then you need to do two things:

1. Set statusBarProvider to true in your init_options
2. Add a callback for metals/status

Then for every other thing that we are adding, we are having to do the same thing.

When I originally started this plugin I envisioned not have a "do this one thing to get started" like there is in coc-metals, but rather instructions in how to tie everything together to hopefully utilize a bunch of other great plugins. However, I'm quickly realizing that we are going to end up creating quite a few callbacks, a lot of the init_option need to be set since the configuration in nvim-lsp is the very base just for Metals to work correctly for a user out of the box without nvim-metals. All that to say, I'm leaning more towards a simple way to add in nvim-metals, and have it do the necessary configuration without having a user add in 30 lines (apart from mappings of course). I think you also had a good point @landerlo about adding a new feature. Ideally a new feature can be added without the user having to adjust their setup.

Approaches

1. Just handle the entire installation of Metals, bootstrapping etc. This would rid the need of nvim-lsp at all. However, this would be the most work, and I'm not sure I want this. Just adding it here as an approach

2. `Maybe instead of having to using the lua << whole setup code, have something like setup_nvim_metals { myOverrides }

3. Something completely different?

Search terms

setup

[mfussenegger]

For nvim-jdtls I ended up going with 1), although I initially was also opposed to doing that.

You can have a look at the change here

In the end it wasn't as much logic as I had thought.

[ckipp01]

Thanks for linking that @mfussenegger! That's really helpful. That's also the route I'm planning on taking as well now when I get some time.

[ckipp01]

So I wanted to do a quick update on this as it's been quite some time since I've had time to dig back into nvim-metals. This weekend I've been able to put some effort into the following:

No longer requiring nvim/nvim-lspconfig

The reason for me doing this is twofold. Firstly, there are plans to remove the ability to manage installations from nvim/nvim-lspconfig. Having this automated is quite nice in my opinion, so I'd like to continue having this feature in the future. More importantly, if you use nvim-metals, you need to override a bunch of handlers, add in init_option and have about 50 lines of configuration to get started. I really dislike this. Locally I have a working POC that no longer requires nvim-lspconfig. It's largely modeled after @mfussenegger 's nvim-jdtls, so a big 🙏🏼 to you for the examples. This simplifies the setup greatly. Now the setup to start using nvim-metals (when this is merged) will simply be:

if has('nvim-0.5')
  augroup lsp
    au!
    au FileType scala lua require('metals').initialize_or_attach({})
  augroup end
endif

The initialize_or_attatch manages the creation or the attaching to an existing server, and the full initialization process. It's much cleaner now, and I'm quite happy with it.

Fully managed installation and update process

I'm currently working on this at the moment. One of the important parts of the Metals plugins so far, is that we wanted an easy way to install and update Metals. In the VS Code extension, coc-metals, and the sublime plugin it was decided to just include an executable coursier that is used to bootstrap metals. For nvim-metals I'm torn between just requiring the user to have Coursier installed on their machine locally and then still managing the installation with the assumption that Coursier or cs is available on the system (this is similiar to how it is on nvim-lspconfig at the moment) or including the executable coursier in the plugin and fully managing everything. I'd like to just keep the plugin as slim as possible, but I also know that having the executable in the plugin might be convenient for new Scala users that may not have Coursier installed. My worry about having the coursier executable as part of the plugin may be totally unfounded, so I'd appreciate any input people have on this.

I'd honestly love to just leave the installation up to a package manager, coursier, but I don't want the user to have to deal with the full

coursier bootstrap \
  --java-opt -Xss4m \
  --java-opt -Xms100m \
  org.scalameta:metals_2.12:0.9.8 \
  -r bintray:scalacenter/releases \
  -r sonatype:snapshots \
  -o /usr/local/bin/metals -f

They can use cs install metals if they only want the latest, but I would want an easy way for a user to just be able to cs install metals --snapshot to get the latest snapshot or a specific snapshot somehow. I think if it was that simple, I'd go this route.

Since it's not, I'm currently leaning towards including the executable since it will make the process much more polished. Plus it allows for more fine grained control over things like being able to set a specific Java version for Metals which may be different then what is being used if you just use the bootstrapped metals from coursier. Plus this will also then be very close to inline with how it's done in coc-metals allowing for many of the same configuration options.

Expect in the next couple weeks to see all of this land in master with some other cool updates.

EDIT: After playing around some more, part of me is starting to believe that sort of a hybrid approach may be best. Basically, the user needs to have coursier installed. If it's not installed, the user will get a warning to install it. Once installed, if g:metals_server_version is defined it will pull from there, if not it will just default to the latest stable. This is similar to how it works now in nvim-lspconfig. Where it differs is that it will just be automatic. If there is a new version out, it will automatically be downloaded unless you have a hardcoded version. It will then just be kept in cache and used to start the server. If you desire a different java version to run, you will just need to manage it via java.

[landerlo]

+1. We can have a variable for the metals executable that way you can change the command the location, etc. Also when you are not installing downloading metals automatically.
How do you plan the automatic part of having metals? Requiring calling a command like metals-install or by checking if the version is there on startup?
I personally would prefer to have the fastest startup possible, just starting the metals process, so the automatic installation calling coursier could be triggered if metals command fails to provide a more out of the box experience, but I think involving coursier when starting metals could add a non negligible overhead, even with the coursier binary executable as it involves name resolution, etc.

[ckipp01]

+1. We can have a variable for the metals executable that way you can change the command the location,

Actually I think I want to restrict this, and I'd rather we fully control this. Basically, I'd rather that we always just use metals as the command, and that's all nvim-metals knows about, and it's up to the user to ensure that it's on their $PATH, or we control where it's at and provide a simple way to update/change the version.

The way I have it now is that the user can just set the following:

let g:metals_server_version = '0.9.8+2-06f59123-SNAPSHOT'

Now when then open up a Scala file that doesn't have a server attatched to it yet, it does a quick bootstrap.

  if util.has_bins('cs') then
    coursier_exe = 'cs'
  elseif util.has_bins('coursier') then
    coursier_exe = 'coursier'
  end

  assert(coursier_exe,
         'Coursier must be installed to use nvim-metals. You can find installation instruction here: https://get-coursier.io/docs/cli-installation')

-- some stuff in the middle here

  local nvim_metals_cache_dir = util.path.join {fn.stdpath('cache'), 'nvim-metals'}
  local metals_bin = util.path.join {nvim_metals_cache_dir, config.name}

  if not uv.fs_stat(nvim_metals_cache_dir) then
    os.execute('mkdir -p ' .. nvim_metals_cache_dir)
  end

  local get_cmd = string.format(
                      '%s bootstrap --java-opt -Xss4m --java-opt -Xms100m org.scalameta:metals_2.12:%s -r bintray:scalacenter/releases -r sonatype:snapshots -o %s -f',
                      coursier_exe, server_version, metals_bin)

  vim.fn.system(get_cmd)

  config.cmd = {metals_bin}

I understand the concern about the fast startup speed. However, if the user has cs installed, then I think this is pretty acceptable:

 time cs bootstrap -r sonatype:snapshots org.scalameta:metals_2.12:0.9.8+2-06f59123-SNAPSHOT -o metals -f
Wrote /Users/ckipp/Documents/scratch-workspace/metals

________________________________________________________
Executed in  201.12 millis    fish           external
   usr time  183.88 millis  100.00 micros  183.78 millis
   sys time  111.44 millis  470.00 micros  110.97 millis

I'd be more in favor of sacrificing a couple houndred millis to ensure no matter what when they start, they are getting what they have set. What do you think? If this is with a cold JVM and coursier, then yea, this is like 4-5 seconds, which isn't ideal.

[ckipp01]

One other option, which I alluded to is to just give nvim-metals the metals, and ensure the user is using Coursier to manage installation. For most users that want to stick on stable, this is as easy as cs install metals. Done. If they want a specific version cs install metals:0.9.7. I'm playing around now with making sure that they could also do cs install metals:latest.release which would give them the latest snapshot. If this is possible, maybe this is even better than doing it the way I have outlined up above. However, up to this point I'm unable to get that to work.

[ckipp01]

Alright, I keep going back and forth, but after writing this I started paying a bit of attention and unless I'm crazy, I can totally tell a difference even with cs. There is a slight pause that I also don't want. Maybe instead it'd be best to detect if there is a metals. If not, then prompt the user to do a :InstallMetals command which will install it for them. Then just skip the bootstrap no matter what when you open a file, which will keep it snappy.

[landerlo]

I think involving coursier un bootstrap will risk non deterministic delays, depends on refresh settings etc, as there is some version resolution, etc. This is fast with cs but I suspect the delay is not acceptable if we don't want to add startup overhead. Doesn't Lsp detect the client command has failed and that exception can trigger an error message or the install? I think we can keep it simple. It's quite a simple prerequisite to document doing cs install metals.

[landerlo]

Thank you for starting looking into this, something I had planned to do. It's unclear to me that lspconfig adds any value and having an opinionated way of doing things tailored to our case makes things much simpler. Same with a lot of the handlers and central configuration stuff.

The client should register automatically for the diagnostic stuff, and for the textDocument/defintion etc. requests it's simpler to use something like I proposed in neovim/neovim#12277 (comment)

Maybe I missed recent development but I don't think the nvim-lsp has improved much in that sense, there's still discussion how to configure different UIs and custom workflows like with codeActions.

Those calls are user initiated requests and the callback is just to make an RPC async, we should pass the callback in the request, and then composing calls with FP combinators to model a workflow.

Nvim-lsp has had very powerful building blocks for a long time, but then there is the expectation that it's going to provide a full package with allowing all possible configuration that complicates things with many changes impacting the adoption IMO.

[ckipp01]

This post will serve as a migration guide for anyone after #37 is merged. Prior to this, a setup would be quite involved with overriding a bunch of handlers and init options. For those of you that have been using nvim-metals up until this point, the following steps will apply to you.

• Remove the Metals installation that you installed with nvim/nvim-lspconfig. If you don't know where this was installed, you can do a :LspInstallInfo and it will show you the path of where it's installed. Remove this. After this you should be able to do a LspInstallInfo command and just see an empty {}.
• Remove all of the old manual settings of init options and callbacks, and replace it with the new instructions which will be both in the help docs and the readme. For example, an old configuration would look something like this:

  local lspconfig = require'lspconfig'
  local metals    = require'metals'
  local setup     = require'metals.setup'
  local M         = {}

  M.on_attach = function()
      " Note that both of these are for external plugins
      require'completion'.on_attach()
      setup.auto_commands()
    end

  lspconfig.metals.setup{
    on_attach    = M.on_attach;
    root_dir     = metals.root_pattern("build.sbt", "build.sc", ".git");
    init_options = {
      -- If you set this, make sure to have the `metals#status()` function
      -- in your statusline, or you won't see any status messages
      statusBarProvider            = "on";
      inputBoxProvider             = true;
      quickPickProvider            = true;
      executeClientCommandProvider = true;
      decorationProvider           = true;
      didFocusProvider             = true;
    };

    handlers = {
      ["textDocument/hover"]          = metals['textDocument/hover'];
      ["metals/status"]               = metals['metals/status'];
      ["metals/inputBox"]             = metals['metals/inputBox'];
      ["metals/quickPick"]            = metals['metals/quickPick'];
      ["metals/executeClientCommand"] = metals["metals/executeClientCommand"];
      ["metals/publishDecorations"]   = metals["metals/publishDecorations"];
      ["metals/didFocusTextDocument"] = metals["metals/didFocusTextDocument"];
    };
  }

The same exact settings from above can now just be replaced with the following

:lua << EOF
  metals_config = {}
  metals_config.handlers = {}
  metals_config.init_options = {}

  metals_config.on_attach = function()
    require'completion'.on_attach();
  end

  metals_config.init_options.statusBarProvider = 'on'
EOF

if has('nvim-0.5')
  augroup lsp
    au!
    au FileType scala lua require('metals').initialize_or_attach(metals_config)
  augroup end
endif

• If you don't use nvim/nvim-lspconfig for other servers, you can just remove the plugin. If you are still using it, as long as you don't try to use it for Metals, it will be fine to have them both installed together.

[ckipp01]

Seeing that the decisions on how this will all be done have now been made, I'm going to go ahead and lock this.