OtterQuic tag-based message passing protocol proposal.

[KTSnowy]

1. Overview

OtterQuic is a message passing protocol designed to be compatible with the asynchronous messaging features introduced in the new COBOL 2023 standard (ISO/IEC 1989:2023).

Even though the standard leaves most of the messaging passing system and how it works as an "implementor-defined" aspect, there are still a couple of specific requirements in the specification. These are the message tag object, the MCS (Message Control System), the capability to send messages both locally and remotely, and the suspension of execution while running the SEND and RECEIVE statements.

During Otterkit's development and while discussing how the message passing system should be implemented we concluded that existing protocols and message passing solutions would not be enough to fully conform to the requirements from the COBOL standard and attempting to force existing protocols or solutions to fit these requirements could end up with a possibly broken and inadequate system being used for an important core language feature.

WebSockets was considered due to it having persistent bidirectional connections but was ruled out due to it requiring a TCP connection (which brings its own problems), an HTTP upgrade handshake for every connection, possible issues with multiplexing support, and for also bringing its own scalability issues.

WebTransport was briefly considered, but was ruled out due to lack of general support, lack of production-ready server libraries and because it still requires a full HTTP server (bringing its own problems) to work.

Ultimately we decided to define a new protocol around QUIC that conforms to the requirements from the COBOL 2023 standard for async messaging.

2. Terminology

• MCS or Message Control System: Is an independent process responsible for sending and receiving messages exchanged between local and/or remote run units.

• Message server: Is a run unit that receives a request from a requestor and returns a message back to that requestor.

• Requestor: Is a run unit that makes a request to a message server to receive a message back from that message server.

• Message Tag: Is a unit of data that specifies the requestor or sender of a message and any extra information about the message itself.

• Run unit: Is a runtime entity that functions independently at execution time (meaning that each run unit is an independent process).

• QUIC: Is a general-purpose multiplexed transport layer network protocol built on top of UDP.

• UTF-8: UCS Transformation Format 8 or just UTF-8 is a variable-length character encoding standard used for electronic communication, defined by the Unicode Standard

3. Goals and Technical Requirements

Primary goal is to define a protocol suitable for being used in the runtime library for Otterkit (and for other COBOL vendors if they choose to use it) to provide the asynchronous messaging features required by the ISO/IEC 1989:2023 COBOL standard.

The protocol specification has to be language, architecture and CPU independent, meaning it cannot explicitly or implicitly depend on the features of a specific programming language, CPU architecture or CPU manufacturer in order to be implemented.

The protocol has to be an application layer protocol. It has to be built around the QUIC network protocol and will use QUIC for all MCS connections, run unit connections, and for sending and receiving messages both for local and remote requests.

The protocol has to allow persistent bidirectional communication. Message servers will have to maintain a persistent bidirectional connection with the MCS process which will be done through a QUIC connection.

The Message Control System is part of this protocol specification and can be considered an OtterQuic server (OtterQuic MCS). Message servers and requestors are both OtterQuic clients, with message servers having persistent connections to the MCS and registering unique identifiers with it in order to easily allow senders to send message requests to servers.

The MCS has to be capable of quickly finding to which run unit or message server a message tag should be sent to. Any message sent has to first go through the MCS which will then handle the actual exchange and sending of the message to the destination.

The MCS has to be capable of sending and receiving messages remotely, meaning that it has to be capable of temporarily opening a peer connection into a remote MCS to exchange remote messages. These remote connections should be secure and ideally authenticated in order to avoid attempts from malicious actors to open unauthorized connections to an MCS instance.

The protocol itself and the OtterQuic MCS server have to be performant enough to be used for local messages servers and reliable enough to send and receive remote messages without constant disconnections or information loss issues.

The protocol specification has to define consistent message tag contents that can be easily read by the MCS without too much performance overhead, it also has to define any other required communication tags and their contents which will be used between message requests. These communication tags are only to be sent to establish connections, close connections, check if a connection is alive, check the validity of a connection request, or for MCS authentication purposes, and are not meant to be sent or received as an actual message tag to the clients.

The protocol has to accept incoming OtterQuic connections only, any incoming request from another protocol has to be refused and the MCS is not required to send any error messages or responses that conform to the incompatible incoming request's protocol.

The protocol is required to refuse incoming remote OtterQuic connections that use a protocol version that does not match the version used by the local MCS. For example, if an incoming remote MCS request uses version OQ01 and the local MCS uses version OQ05 then the local MCS is required to refuse and immediately close the connection.

If authentication is enabled the local MCS is required to immediately close and refuse any incoming connections that fail to correctly authenticate. Enabling authentication has to be opt-in (developer choice), and will require an additional MCS communication tag for authentication purposes.

The implementation is not required to be able to fetch (outgoing request) remote HTTP resources. The implementation is free to include an HTTP fetch extension if needed but such extension is required to wrap the HTTP response in a message tag before sending it to the destination run unit and should never send raw HTTP responses through OtterQuic connections.

4. Proposed OtterQuic Protocol Design

The proposed protocol is an application layer message passing protocol over QUIC, and because of that it only handles what happens immediately after a successful QUIC handshake and before the QUIC connection is closed. The proposal does not directly modify or extend any of QUIC's functionality, and what happens before the handshake or after the connection is fully closed is out of the scope of this proposal.

4.a Version Request Tag

Immediately after a successful QUIC connection with the Message Control System the requestor has to send a 4 bytes version request tag.

Version Request Tag Text Format

OQ01

Version Request Tag Hex Bytes Format

0x4F 0x51 0x30 0x31

The first two bytes are used for the protocol magic bytes 0x4F and 0x51, and the last two bytes are for the protocol version number from 01 to 99. Each byte has to use the UTF-8 encoding for each character of the text format respectively. Other encodings are not supported.

The version bytes sent from the requestor has to match exactly with the version bytes from the MCS. If the version bytes do not match exactly for any reason or if any other data format is received instead of the version bytes, the MCS has to send an ERR:EC-MCS-INVALID-TAG request tag back to the requestor and the connection is immediately closed.

4.b Authentication Request Tag

Immediately after a successful version exchange and if remote request authentication is enabled, the requestor has to send the authentication key for the Message Control System in an authentication request tag format.

Authentication Request Tag Example

KEY:ro9MEl5BI0Gbhn1rf5M7WgZoPZtz2CuUS-eqN8qo7bVg

The authentication tag has to start with the prefix KEY:, otherwise the tag is considered invalid and the MCS has to send an ERR:EC-MCS-INVALID-TAG request tag back to the requestor. The connection is immediately closed.

If the key contained in the authentication request tag does not match exactly the authentication key for the MCS then the key is considered invalid and the MCS has to send an ERR:EC-MCS-INVALID-TAG request tag back to the requestor. The connection is immediately closed.

The authentication key (or multiple keys) is configured by the developer using the Message Control System's configuration file. The configuration file has to be capable of accepting both hard coded keys and an environment variable format that allows the MCS to fetch an authentication key from a developer defined environment variable name.

This request tag is only used for incoming remote requests. Local message servers and local requestors are not required to authenticate and should not send an authentication request tag to the local MCS instance.

4.c Message Tags

Immediately after a successful version exchange (and an authentication key exchange if required by the MCS) the requestor has to send the message tag content associated with the request.
The message tag content is separated into 4 parts: The protocol method, the message origin, the message server route, and the text data being sent.

Message Tag Methods

METHOD:MCS

METHOD:SEND

METHOD:REGISTER

METHOD:MCS has to be used only for sending error and exception messages from the local or a remote MCS back to a requestor. Example of an exception condition is if a local message server name does not exist.

METHOD:SEND has to be used only for sending message tags (and their text data) between requestor run units and message servers.

METHOD:REGISTER has to be used only for registering local message servers with the local MCS, enabling them to be discovered by requestor run units by using the name of the message server.

Message Tag Origin Formats

ORIGIN:{REMOTE-URL}

ORIGIN:LOCAL

ORIGIN:{...} has to be used only for specifying the MCS endpoint. The origin for local message servers and local requestors has to always be ORIGIN:LOCAL, and the origin for remote MCS endpoints has to be a valid URL that can be used to send messages back to the origin if needed.
The URL used for the MCS endpoint has to be developer defined for each individual MCS instance in their own configuration file.

Message Tag Route Format

ROUTE:{MESSAGE-SERVER-NAME}

ROUTE:{...} has to be used only for specifying which message server the message tag is to be sent to. The message server name specified must uniquely identify an already registered message server in the local or remote MCS instance.

Message Tag Data Format

DATA:{MESSAGE}

DATA:{...} has to be used only for sending UTF-8 encoded alphanumeric data between message servers and requestors. This is the only part of the message that the application itself will receive as readable and writable message data (minus the DATA: prefix). All other parts of a message tag are instead stored in an appropriate message tag object in the application's memory storage.

The text data being sent has to use the UTF-8 encoding for the entire message data. Other encodings are not supported.

All message tag bytes received after the DATA: prefix must be considered UTF-8 encoded alphanumeric data and must not be interpreted or executed as anything other than pure UTF-8 encoded alphanumeric text information.

Message Tag Null Text Format

NULL

Message Tag Null Hex Format

0x4E 0x55 0x4C 0x4C

The NULL message tag format has to be used only as the default contents or a newly created message tag object, or to allow a message server to receive a message from any requestor origin and route.
The NULL message tag content has to be a UTF-8 encoded alphanumeric representation of the word "null". It must not use a language specific representation of null and must not contain null bytes such as a UTF-8 NULL character.

To conserve memory usage, the implementation might choose to cache or preallocate a static instance of a NULL message tag and distribute a pointer or reference to the same static instance when an instance of a NULL tag is required.

Full Message Tag Example

METHOD:SEND ORIGIN:LOCAL ROUTE:fetch-customers DATA:{data-query...}

The above is an example of a full message tag, the message specifies that the tag was sent by a local requestor, being sent to the fetch-customers message server. The message data contained in developer defined, and ideally should only contain information that the local message server is able to parse.

SOURCE: OtterQuic Design Doc