Better autocomplete support

[garazdawi]

Describe the bug

I would like to have better auto-complete support, such as what is available in the Erlang shell.

Expected behavior

Today you can call this in an Erlang system:

1> edlin_expand:expand(lists:reverse("gen_tcp:connect(localhost,80, ["),[]).
{no,[],
    [#{options => [highlight_all],
       title => "typespecs",
       elems =>
           [#{options => [{highlight_param,3}],
              title => "gen_tcp:connect(Address, Port, Opts)",
              elems =>
                  [#{options => [{hide,title}],
                     title => "types",
                     elems =>
                         [#{options => [{separator," :: "},{highlight_all}],
                            title => "inet:inet_backend()",
                            elems => [{"{inet_backend, ...}",[]}]},
                          #{options => [{separator," :: "},{highlight_all}],
                            title => "gen_tcp:connect_option()",
                            elems =>
                                [{"{fd, ...}",[]},
                                 {"inet",[]},
                                 {"inet6",[]},
                                 {"local",[]},
                                 {"{ifaddr, ...}",[]},
                                 {"{ip, ...}",[]},
                                 {"{port, ...}",[]},
                                 {"{tcp_module, ...}",[]},
                                 {"{netns, ...}",[]},
                                 {"{bind_to_device, ...}",[]},
                                 {"{active, ...}",[]},
                                 {"{buffer, ...}",[]},
                                 {"{debug, ...}",[]},
                                 {"{delay_send, ...}",[]},
                                 {"{deliver, ...}",[]},
                                 {"{dontroute, ...}",[]},
....

You get a list of all the different options that gen_tcp accepts, and then if you do:

2> edlin_expand:expand(lists:reverse("gen_tcp:connect(localhost,80, [{active,"),[]).
{no,[],
    [#{options => [highlight_all],
       title => "typespecs",
       elems =>
           [#{options => [{highlight_param,3}],
              title => "gen_tcp:connect(Address, Port, Opts)",
              elems =>
                  [#{options => [{hide,title}],
                     title => "types",
                     elems =>
                         [{"true",[]},
                          {"false",[]},
                          {"once",[]},
                          {"range(op (- 32768), 32767)",[]}]}]},
            #{options => [{highlight_param,3}],
              title => "gen_tcp:connect(Address, Port, Opts, Timeout)",
              elems =>
                  [#{options => [{hide,title}],
                     title => "types",
                     elems =>
                         [{"true",[]},
                          {"false",[]},
                          {"once",[]},
                          {"range(op (- 32768), 32767)",[]}]}]}]}]}

you get only the things allowed as {active, option. The data by this is triven by the function specs found in the system. The return format is documented here: https://www.erlang.org/doc/apps/stdlib/edlin_expand.html

I would like ELP to either use the output from edlin_expand directly to provide autocomplete results, or that we create a new API in Erlang/OTP that ELP can use. Elixir has something call Code.Fragment
which works similarly to what I have in mind.

What do you think?

/cc @frazze-jobb

[robertoaloi]

Hi @garazdawi !

It's an interesting idea. I think we can split the discussion in two parts:

1. edlin provides smart parameter completion based on spec information, but ELP does not. Can we get that?
   I can see the advantage and, in principle, it sounds doable (to a certain extent).

2. Can we somehow reuse the edlin logic to power ELP auto-completion?

Here things get a bit trickier. ELP is mainly implemented in Rust and, even if it comes with an Erlang side-car where we can run Erlang code, performing actions natively has its advantages. Furthermore, ELP mostly acts on source code, while edlin seems to operate on the doc chunks, which would require the code to be compiled.

I quickly went through the edlin modules and, essentially, edlin replicates some of the responsibilities of a LSP server and redefines certain features, such as signature help, providing its own version of it.

Did you ever consider having, instead, the shell itself acting as a LSP language client? As an example, a "TAB" in the shell would trigger a textDocument/completion request to the server, which would then take care of the rest and return a number of completion results to the client, for printing.
The advantage would be to use an established protocol, at which point we would be re-using the same logic across the shell and the IDE.

After all, the language server works very similarly to edlin when it comes to completions: first we build a Context for the auto-completion, then we match on the context to return relevant results.

edlin does more or less the same: first it extracts the context, then it provides completions based on it.

I don't know if there are any other languages doing this, but it would be a fun experiment to do.

[garazdawi]

Here things get a bit trickier. ELP is mainly implemented in Rust and, even if it comes with an Erlang side-car where we can run Erlang code, performing actions natively has its advantages.

You will have to decide where you want to draw that line. When working in the shell I find that getting the type data as autocompletions greatly decreases the amount of times I need to switch to the docs. Maybe eqwalizer has some data that you can use if you don't want to use edlin?

Did you ever consider having, instead, the shell itself acting as a LSP language client? As an example, a "TAB" in the shell would trigger a textDocument/completion request to the server, which would then take care of the rest and return a number of completion results to the client, for printing.

While we could do that we would always need some bare minumum of things to work as a language server is not always available.

Anyway, I opened this issue in order to make you aware that this API exists and also to see if you could use it and if any changes would be needed.

[robertoaloi]

Maybe eqwalizer has some data that you can use if you don't want to use edlin?

From what I see edlin extracts most of the information from the specs, so we should be able to do the same (see my first point).

Anyway, I opened this issue in order to make you aware that this API exists and also to see if you could use it and if any changes would be needed.

Yes, and it's really appreciated! I will hopefully play with this soon, most likely after the Christmas break.