Use Wasm for plugin mechanism

[eduardomourar]

Currently, plugins for Smithy can only be developed in Java, but I would like to use another language (e.g. TypeScript) instead.

By using a different plugin system based on Wasm, any language that can target WebAssembly can be used to extend Smithy.

More details here: johnstonskj/rust-atelier#40

[mtdowling]

Wasm is interesting, but it still seems very early for it as a tool outside of browser, so I don't think it's the right for Smithy extensions right now. A process-based model for extending Smithy by calling another process could make sense (e.g., Smithy's Java implementation passes the JSON AST to another process, that process returns an updated AST).

[eduardomourar]

Although WebAssembly is in its early days, it is very powerful with the addition of WASI that allows filesystem access, for instance. Basically, the Wasm plugin approach would follow the same as the process-based one, but instead of just sending the AST back and forth, it could additionally receive functions used within the Java core application. The communication would be a lot richer such as the one used by the Istio team (more details here), where they allow the client to perform certain actions from the host/runtime.

The good thing with Wasm is the fact that it is a single binary file, while the process-based option it would requires us to include in our Smithy CI/CD the whole application running the external process.

[mtdowling]

Yeah, a process based approach would not be as efficient. It would be reserved for when you want to write a plugin in a different language. If efficiency is important, plugin authors can always write their plugins in Java when using Smithy's reference Java implementation.

Has WASI reached 1.0 yet? Do you know how we'd embed Wasm in Java? Is it done by embedding V8, or is there a lighter-weight solution?

The good thing with Wasm is the fact that it is a single binary file, while the process-based option it would requires us to include in our Smithy CI/CD the whole application running the external process.

I'm not sure I follow this part. If Smithy's Java implementation supported Wasm, you'd still need the Java implementation to load and invoke the Wasm binaries. You could also create standalone binaries if Smithy offered a process based solution, so I don't see the distinction here as it relates to CI/CD.

[eduardomourar]

Has WASI reached 1.0 yet? Do you know how we'd embed Wasm in Java? Is it done by embedding V8, or is there a lighter-weight solution?

WASI is not in a stable version yet, but you can check here how far each individual proposal is.

My suggestion would be to use Wasmtime although I have not used with Java (https://github.com/kawamuray/wasmtime-java) so the performance might be suboptimal.

[mtdowling]

Another option is GraalVM, which supports lots of languages, including JavaScript, Wasm, and LLVM.

[eduardomourar]

Yes, that is also a possibility. I believe Wasmtime is lighter and more performant than GraalVM, though. It also depends if there are plans to change the host platform in the future.

[mtdowling]

I'm going to go ahead and close this since a WASM based plugin system isn't something we can commit to in 2022. There are other issues that would hold us back from adding this like that WASI isn't stable and there's a lot of research that would need to go into seeing how well this would work in Java and what languages we'd actually be able to support in WASM.

[eduardomourar]

Interesting to note that Wasmtime has reached v1.0 (announcement here) and WASI support for Java has been implemented here: https://github.com/kawamuray/wasmtime-java.