Multiple entries in eglot-server-programs for the same mode

[mkcms]

I think it would be convenient if we allowed users to add multiple entries for the same major mode to eglot-server-programs, and then tried each one in order until we find one for which an executable exists For example: (setq eglot-server-programs '((c-mode . "cquery") (c-mode . "clangd")))

Or alternatively: (setq eglot-server-programs '((c-mode . (or "cquery" "clangd"))))

[joaotavora]

Actually, @mkcms we could enhance this to also allow multiple servers to manage the same buffer.

[mkcms]

@joaotavora
Interesting. But in what scenario would that be useful? I can only think of web development for editing HTML templates. I'm not sure if there are any servers that work like that though?

I was actually planning to extend the way eglot chooses servers for managing buffers like that:

1. Add ability to share one server in multiple major modes:

   ((c-mode c++-mode :shared) . "clangd")

   I already implemented this and it worked OK (instead of having just a single major-mode slot in eglot-lsp-server I changed it to a list - major-modes)

2. Add a new variable so that you can specify contacts per-project in dir-locals. Of course, you can do that now, but you have to override the entire eglot-server-programs variable.

   This would be useful for example for the eclipse server. It can work with both xml and java files, but the user probably doesn't want to use the eclipse server for all xml files in non-java projects. So they could add to dir-locals:

   ((nil
      (eglot-local-contacts
       (java-mode nxml-mode :shared)
       . eglot--eclipse-jdt-contact)))

3. Allow multiple entries in eglot-server-programs - this is what this issue is about.

All of those things are rather easy to implement, and I already implemented no. 1. I just wanted to have them all before making a PR. Do you think those they are useful?

[joaotavora]

@mkcms

I am weary of adding syntax to eglot-server-programs. That said

• Your (1) doesn't look terrible, but why don't we make it the default (and avoid :shared)? IOW when would we ever want different files of different major-modes in the same project, that can be managed by the same server, to not be managed by the same server? If we do need that, then we can always say the way to do it is by adding a separate entry.

• In (2) Overriding eglot-server-programs doesn't sound very bad, especially if you can somehow tap into the default value.

• In (3), I wanted to transform this issue into a discussion of what the semantics of this repetition should be. You proposed a "find the first matching server" strategy, which would work with the current infrastructure. I propose a "find all matching servers and start them all", but this requires much deeper work.

[mkcms]

* Your (1) doesn't look terrible, but why don't we make it the default (and avoid `:shared`)? IOW when would we ever want different files of different major-modes in the same project, that can be managed by the same server, to _not_ be managed by the same server?  If we do need that, then we can always say the way to do it is by adding a separate entry.

I will open a PR for that.

* In (3), I wanted to transform this issue into a discussion of what the semantics of this repetition should be.  You proposed a "find the first matching server" strategy, which would work with the current infrastructure

Alternatively, could we still use the current infrastructure, but just ask which server to use?

* I propose a "find all matching servers and start them all", but this requires much deeper work.

I don't get that though. Would multiple servers manage that buffer, or would they just be started to see if they work or something? Would e.g. a definitions request ask all servers in current buffer and then merge the results into one list?

[joaotavora]

I don't get that though. Would multiple servers manage that buffer, or would they just be started to see if they work or something? Would e.g. a definitions request ask all servers in current buffer and then merge the results into one list?

Well yes. At least this is what I think makes sense. But it's better to hold off this development for a while, since the changes would be truly drastic. Everywhere we use server, singular, we would probably have to use servers, plural. So:

• I'll check what lsp-mode does with this, since it recently announced "support for multiple servers".
• It might be more interesting to take a completely different approach: create a new package, say eglot-meta.el that is an "proxy" LSP server to multiple servers. This fake server would multiplex every request from the client into multiple requests to its real servers, and return the first response by any of those servers. Notifications and server requests would be re-numbered and passed to the client, and client responses would also be renumbered and returned to the server. The transport between eglot.el and eglot-meta.el needn't be stdio/tcp if we leverage jsonrpc.el's API, it can be just a buffer.

[mkcms]

But it's better to hold off this development for a while,

So should I look into implementing 'asking the user which server to start'?

* It might be more interesting to take a completely different approach: create a new package, say `eglot-meta.el` that is an "proxy" LSP server to multiple servers.

That's pretty interesting. To take this a bit further, it could just be a python script, so that all LSP clients can use it, IIUC.

[joaotavora]

So should I look into implementing 'asking the user which server to start'?

How are you planning to coordinate that with the existing interactive behavior of eglot--guess-contact?

That's pretty interesting. To take this a bit further, it could just be a python script, so that all LSP clients can use it, IIUC.

Yes, indeed you could very well write this in whatever language you prefer.

[joaotavora]

Yes, indeed you could very well write this in whatever language you prefer.

And python is probably a good choice given its popularity...

[mkcms]

How are you planning to coordinate that with the existing interactive behavior of eglot--guess-contact?

Good question. I guess there's no option but assume that functional contacts are always present? E.g. include the contact as a choice, and call it only after it was chosen by the user.

[mkcms]

How are you planning to coordinate that with the existing interactive behavior of eglot--guess-contact?

Above I was talking about functional contacts. In general, is there anything special to consider?

1. If there are multiple servers which can be started, ask the user which one to start
2. If there is exactly one server which can be started, start it.
3. If there are no known servers, go back to the old behavior - ask the user for path to the program/host:port.

[joaotavora]

Above I was talking about functional contacts. In general, is there anything special to consider?

It has to generally make sense :-)

If there are multiple servers which can be started, ask the user which one to start

OK.

If there is exactly one server which can be started, start it.

OK. Unless the user explicitly use C-u M-x eglot, right, in which case you should prompt for details.

If there are no known servers, go back to the old behavior - ask the user for path to the program/host:port.

OK

In general, I think the best way to do this is to rewrite the current eglot--guess-contact very slightly, so that it returns the cdr of the eglot-server-programs list that wasn't analysed. Then to do 1, call it repeatedly, non-interactively, until you collect every server that can be started non-interactively.

You are also looking at a significant overhaul of the eglot docstring, and/or the eglot-server-programs docstring. Please don't underestimate this effort. They are already quite difficult to parse as is...

[joaotavora]

In general, I think the best way to do this is to rewrite the current eglot--guess-contact very slightly

Then again, this isn't strictly true. If you want to rewrite more deeply and you can make it readable (and not enormous), that's also OK.

Anyway, I would start with the docstrings and move on from there.

[mkcms]

You are also looking at a significant overhaul of the eglot docstring, and/or the eglot-server-programs docstring. Please don't underestimate this effort. They are already quite difficult to parse as is...

OK, I will do that. However probably not soon, because I'm pretty busy now. I probably won't be very active for a while.

[casouri]

If there are multiple servers which can be started, ask the user which one to start

I suggest to keep user interaction out of eglot--guess-contact and add another command to choose from different servers interactively. If eglot--guess-contact contains interactive parts, what if:
I have a list of Org files with python code in them
→ I export them automatically with some lisp code
→ Org mode tries to syntax highlight python code, and uses python-mode
→ eglot asks what server I want to use, or it errors
→ I don't want that.

To be honest I'm not very familiar with eglot or Org Mode, so this might not be the case. But something similar happens when I enabled lsp-mode and tried to export some Org files; and I guessed that this is the cause (because the debug trace has something about lsp-mode and I know lsp-mode sometimes requires you to chose the project root interactively).

[mkcms]

@casouri

→ I don't want that.

You won't get that, unless you do something you shouldn't. If you just use the recommended way of enabling eglot in some mode automatically, e.g. (add-hook 'python-mode-hook 'eglot-ensure), Eglot won't ask you any questions. Even now, if you use java-mode and call M-x eglot it will ask you for path to the server. But if you call eglot-ensure, it won't.

[casouri]

Multiple servers is a cool idea, but how about the original proposal (a list of servers)?

That could be useful for configuring default servers. We can just have a list of servers in the default configuration. And the one that the user installed will be used. This way, users wouldn't even need to customize eglot-server-programs.

[tqchen]

Jump in this thread, to provide a usecase scenario for multiple language servers bought up by
@masahi

We recently build a project specific language server to navigate across FFIs. The idea is that while pyls and c++ works perfectly fine for find definitions within languages, they cannot trace cross language ffis. While that is a hard problem in general, we can build a specific tool for our project which detects certain patterns and tracks calls from python to c++.

This would however, requires the ls client to be able to make use of this project-specific language server along with the generic ones, and combines the result from both to get the final result.

[nemethf]

@tqchen, I think even in your case, one can potentially write a proxy LSP server that combines the result form the generic and the special servers. Then clients can work without modification. This way, only one proxy has to be written, whereas your proposal requires to modify every client.

[joaotavora]

Yep, non-eglot-dependent proxy-server seems like the way to go here, at least on first glance. I'm happy to start that discussion with some knowledgeable people tho.

[joaotavora]

I'm happy to start that discussion with some knowledgeable people tho.

I'm sorry, I see this is more or less a good discussion venue for this already. And my silly wording seems to imply that I think you all aren't knowledgeable. Sorry about that, it doesn't make any sense 🤦‍♂️.

[nemethf]

Then to make the discussion thrive, I'd like to mention that there's an LSP proxy I use all the time, which might be generalized to combine multiple server "backends": https://github.com/sourcegraph/lsp-adapter/

[joaotavora]

might be generalized to combine multiple server "backend

What do you mean generalized? You mean changing it? It... looks dauntingly complex. And needs Go. I was thinking of something Python of maybe Perl based. But I guess if you can make that work, then, why not? Is it free software?

[tqchen]

Proxy server approach sounds good to me, as long as there is a minimum guideline for users to configure it.

IMO It really depends on the level of complexity of the client, in the particular use-case that we need, it would be a matter of change the server to a list, and have a way to combine results(e.g. concat all the lists). So it might not be a bad feature to have on the client side given the minimum complexity. This particular use-case was already supported by the lsp-mode. Although I am not expert on this project so I might be wrong here.

[joaotavora]

So it might not be a bad feature to have on the client side given the minimum complexity.

It is a good feature to have on the client side, but not necessary on the final LSP client editor.

would be a matter of change the server to a list, and have a way to combine results(e.g. concat all the lists

It's more than that. You have to send every request to two servers, combine responses in different ways (first available, wait for all possible). And you have to keep per-server capabilities, too. So it's best to offload that hard work to a multiplexer that the Eglot client (or any other client) sees as just one server.

[joaotavora]

Just another note: you can do all that in Emacs lisp, but you would be limiting yourself to contributors/users in Emacsland (needlessly in my opinion).

[masahi]

I also like the proxy idea and it would be interesting to see a different take on how to approach multi-server scenarios than lsp-mode.

If my understanding is right, this proxy is not eglot or particular server specific, so it can be written once and for all and it would remove the complexity of implementing multi-server handling (be it client or serer side). It would benefit other people working in the same space.

(it reminds me of the fact that the purpose of the LSP is to be a proxy for different languages to different editors, interesting we are talking about another layer of proxying :) )

[joaotavora]

(it reminds me of the fact that the purpose of the LSP is to be a proxy for different languages to different editors, interesting we are talking about another layer of proxying :) )

My thoughts exactly 👍

[nemethf]

[lsp-adapter]

You mean changing it? It... looks dauntingly complex. And needs Go. I was thinking of something Python of maybe Perl based. But I guess if you can make that work, then, why not? Is it free software?

It's under MIT license. I think go is as good as any other language. But this is not my "personal itch", so I don't intend to contribute to a multiplexer proxy. (But that doesn't stop me from speaking gibberish.)

[joaotavora]

I'm closing this as I'm fairly sure this this supported by eglot-alternatives done in context of #688 and #537 (a5b7b7d)

[garyoberbrunner-gpsw]

Just to be clear though (for folks coming here without context), eglot-alternatives does not support multiple servers running at the same time, just an ordered list of alternatives from which it chooses one to actually use. Actually using multiple servers (as supported by lsp-mode, which I'm hoping not to have to switch to) is different. See #1429 for that.