File descriptor setup upon deployment

When being deployed into a Keep, a Wasm workload may be provided with a set of file descriptors to communicate with external entities. In this draft we take into account of the following variants:

• Standard I/O
• Outgoing network connection (TCP, QUIC, ...)
• Incoming network connection (TCP, QUIC, ...)

The simplest form is standard I/O, which wasi-common already provides a way to configure through the Handle trait and stdio methods on WasiCtx. For network connections, there are a couple of options.

Option 1: Let the Wasm workload do the connection setup by itself

This requires the WASI socket API proposal or similar to be merged in the upstreams (WASI and wasmtime).

Option 2: Dynamically inject file descriptors to the Wasm workload

This is probably the most flexible, though it might have security implications. The idea is to make the table used for managing file descriptors public to the WASI host as well as other custom WASI instances. This would require modification to the wasi-common crate something like this.

Still, poll and accept would be missing in this approach, that would be problematic in TLS cases (see option 3.4).

Option 3: Use the existing preopen machinery

Similarly to stdio, wasi-common allows applications to pass in a virtual Handle as a file visible to the Wasm guest, though there is a limitation such as poll_oneoff doesn't work on that handle. As for network connections, outgoing connections (i.e., created with connect) would be simple, while incoming connections (i.e., created with listen and accept) would be tricky because those connections can come at any time.

Option 3.1: Launch a fresh instance of the Wasm workload each time when enarx-wasmldr accepts a new connection

This is simple, though sharing the existing connection(s) among multiple Wasm instances can be tricky.

Option 3.2: Expose listening socket as a directory to the Wasm workload, represent a new connection as a file created under it

This might provide more flexibility, though it require changes in wasi-common because pre-populated virtual filesystem cannot be modified by the host after the Wasm workload launches.

Option 3.3: Expose a listening socket as a regular file, the Wasm workload is responsible for accepting connections

This would eventually invent an HTTP/2 like multiplexing protocol.

Option 3.4: Have the runtime issue listen() and pass the listening file descriptor to the wasm code. The wasm code would then issue either poll() or accept() on the listening socket.

Because we want to support TLS sockets, the configuration file will have to contain the acceptable parameters for the remote identity (i.e. CA cert, etc). Also, poll() and accept() must only return to wasm once a TLS negotiation is already complete.

Additional Considerations

It is our intent only to support TLS sockets. This means that our IO functions have subtly different meanings from their usage with TCP. For example:

• poll(POLLIN) - Under TCP this means that at least one byte is readable. Under TLS this means that at least one block has been successfully decrypted and has remaining bytes to consume.
• accept() - Under TCP this means that TCP negotiation has completed and a new connection is available. Under TLS this means that both TCP negotiation and TLS handshake have completed.

This leads to a situation where a poll() from WASM exits to the runtime, but the runtime itself must issue the poll()/read()/write() syscalls multiple times before returning to the WASM layer. The same is true for accept(). When the WASM layer calls accept(), the wasmldr runtime will be calling the poll()/read()/write() syscalls multiple times across multiple file descriptors before it actually returns to WASM.