diff --git a/0024-off-chain-runtime-logic.md b/0024-off-chain-runtime-logic.md new file mode 100644 index 0000000..ffb714a --- /dev/null +++ b/0024-off-chain-runtime-logic.md @@ -0,0 +1,263 @@ +# ADR 0024: Runtime Off-chain Logic (ROFL) + +## Component + +Oasis Core, Oasis SDK + +## Changelog + +- 2023-11-27: Initial draft + +## Status + +Proposed + +## Context + +Sometimes we may want the runtime compute nodes to run additional off-chain +logic that communicates with the on-chain state securely (e.g. ensuring that the +off-chain logic is being run by the same node operator, is properly attested +when running in a TEE, etc.). + +The off-chain logic may then perform non-deterministic and potentially expensive +things (like remote HTTPS requests or complex local computation) and securely +interact with the on-chain logic via transactions. + +The main use case driving this proposal is support for running attested light +client committees that read and verify information from other chains, then make +this information available to Oasis runtimes with no additional trust +assumptions. + +## Decision + +While similar functionality can be implemented entirely independently on the +application layer (and such solutions already exist), this proposal attempts to +reuse the same security and attestation infrastructure that is already available +for on-chain parts of the runtimes, specifically: + +- Compute nodes and runtime binary distribution and execution can stay the same + as it has been for existing node operators that run the runtimes. Handling of + the off-chain logic part should be done transparently if the runtime provides + it. + +- Existing attestation, consensus and freshness proof flows can be leveraged for + ensuring that the off-chain logic is running in a secure environment. + +One important consideration is also whether to have the off-chain logic part of +the same runtime binary or have it as a completely separate binary running in +its own process. This proposal decides on the latter to ensure that the +off-chain TCB is completely separate from the on-chain TCB. Given that the logic +running off-chain can be much more complex and can interact with untrusted +external services, ensuring this separation is important as a defense-in-depth +measure. + +The proposed architecture extends the composition of the runtime so that it now +contains the following components: + +- **Runtime On-chain Logic (RONL)** is what has existed as the sole runtime + component before this proposal. It contains the logic (and TCB) that is + responsible for executing the deterministic on-chain logic of the runtime. + +- **Runtime Off-chain Logic (ROFL)** is an optional runtime component that may + run in parallel with RONL and is part of its own TCB. It also uses the same + general runtime framework and RHP, but instead of implementing the on-chain + batch scheduling, execution and query methods, it only implements specific + notification hooks that can trigger arbitrary actions. + +Both RONL and ROFL are managed as independent runtimes by the Oasis Node as +host, using the existing runtime host architecture. Failure of ROFL does not +affect RONL which can proceed to run as usual. + +### Attestation + +An assumption made in this proposal is that both RONL and ROFL components are +developed and built together, by the same entity, and are part of the same +release. This means that we can simplify attestation by making RONL being able +to attest ROFL by being aware of its exact identity. + +The idea is that during the release build process, ROFL is built first, its +signer-independent identity (e.g. MRENCLAVE) is measured and included during +compilation of RONL. The signer-dependent part of identity (e.g. MRSIGNER) is +assumed to be the same for both and can be read from trusted CPU state (since it +may not be available during the build process due to offline signing). + +Alternatively, one can imagine a proposal where the ROFL identity is backed by +some sort of on-chain governance process defined in the RONL component. Defining +such a mechanism is outside the scope of this proposal. + +The process for ROFL attestation proceeds as follows: + +1. **Remote Attestation.** The normal runtime attestation flow is initiated by + the host. As a result of this flow, the `node.CapabilityTEE` structure is + generated which includes the remote attestation quote and additional data. + +2. **Node Endorsement.** The host verifies the `node.CapabilityTEE` structure + and if deemed correct, it signs it using the node's identity key and the + following domain separation context: + + ``` + oasis-core/node: ROFL TEE capability + ``` + + The signature is stored in a new structure `EndorsedCapabilityTEE` which is + defined as follows: + + ```go + type EndorsedCapabilityTEE struct { + // CapabilityTEE is the TEE capability structure to be endorsed. + CapabilityTEE CapabilityTEE `json:"capability_tee"` + + // NodeEndorsement is the optional node endorsement signature. + NodeEndorsement *Signature `json:"node_endorsement,omitempty"` + } + ``` + +3. **Updating Node-Endorsed CapabilityTEE in ROFL.** The `EndorsedCapabilityTEE` + is sent to ROFL to be stored and available for establishing secure EnclaveRPC + sessions. + +4. **RONL Verification.** When establishing a new session with RONL, the + endorsed TEE capability is presented during session establishment. RONL + verifies the quote, ensures the enclave identity is one of the known + identities set at compile-time and verifies the node endorsement against the + locally known node identity (both RONL and ROFL must be from the same node). + + If all the checks pass, a secure EnclaveRPC session is established. + +This flow needs to be repeated whenever RAK changes for any reason and also +periodically to ensure freshness (consistent with the quote policy configured +for the runtime in the consensus layer). + +### Updates to the ORC Manifest + +The ORC manifest is extended with a field that can specify extra components +which currently include ROFL binaries in a similar way as we already support +regular runtime binaries (e.g. specifying the executable and SGX metadata). + +### Updates to the Runtime Host Protocol + +This proposal includes some non-breaking updates to the Runtime Host Protocol in +order to support the ROFL component, as follows: + +- **Consensus Block Notification.** No updates are required to facilitate + notifications about consensus layer blocks as this is already handled as part + of the existing RHP flow. The only change is that for ROFL, these + notifications invoke a hook that can be implemented by the runtime. + +- **Runtime Transaction Submission.** A new method `HostSubmitTx` is introduced + which allows ROFL to submit transactions to the runtime. It works by queueing + the transaction in the transaction pool (local queue) for later scheduling. + + ```go + type HostSubmitTxRequest struct { + // Tx is the raw transaction data. + Tx []byte `json:"tx"` + } + ``` + +- **RONL-ROFL Communication.** The existing EnclaveRPC is reused to facilitate + the communication between the two components if/when needed. For this purpose + the endpoint identifier `ronl` is made available in the ROFL host method + handler to address the RONL component. + +### Updates to Runtime Local RPCs + +The ROFL component includes the following default local RPC methods: + +- **Updating Node-Endorsed CapabilityTEE in ROFL.** A new local RPC method + `rofl.UpdateEndorsement` is introduced which allows the node to refresh the + `EndorsedCapabilityTEE` for ROFL. + + ```rust + /// Arguments of the `rofl.UpdateEndorsement` local RPC method. + pub struct UpdateEndorsementRequest { + pub ect: EndorsedCapabilityTEE, + } + ``` + +### Updates to EnclaveRPC RAK Binding + +Version 2 of the `RAKBinding` structure is introduced for establishment of +EnclaveRPC sessions, as follows: + +```rust +pub enum RAKBinding { + // ... previous versions omitted ... + + /// V2 format which supports endorsed CapabilityTEE structures. + #[cbor(rename = 2)] + V2 { + ect: EndorsedCapabilityTEE, + binding: Signature, + }, +} +``` + +Additionally, the relevant EnclaveRPC session implementation is updated to +facilitate thew new authentication mechanism via endorsed TEE capabilities and +the session demultiplexer is updated to support authentication policies on +incoming connections. + +### Updates to the Runtime Host Sandbox + +This proposal updates the runtime host sandbox to support optionally allowing +external network requests. These are then allowed only for the ROFL component +(if any is available for a runtime). + +The following modifications are required: + +- When setting up the Bubblewrap sandbox, `--share-net` is passed to share the + network namespace with the sandboxed runtime. All other namespaces are still + unshared. + +- The runtime loader is modified to accept an additional argument + `--allow-network` which then changes the usercall extension to pass through + any address passed in the `connect_stream` handler. + +### Configuration + +ROFL may require additional configuration which it may do through one of several +ways: + +- **On-chain Configuration.** Configuration for the ROFL component may be stored + in on-chain state. ROFL would then query the current configuration and apply + it locally. + +- **Local Per-Node Configuration.** In case some per-node configuration is + required (e.g. to allow the node operator to override a default), the existing + runtime local configuration mechanism can be used where configuration is + provided as part of the RHP handshake. All configuration for ROFL should be + contained under the `rofl` configuration key. + +### Untrusted Local Storage + +ROFL may utilize the existing untrusted node-local storage to store things like +sealed data local to the node. This store is shared between RONL and ROFL, but +all ROFL keys are transparently prefixed by `rofl.` on the host such that only +RONL can see (but not necessarily read) ROFL's keys but not vice versa. + +### Updates to the Oasis SDK + +A convenient way to develop ROFL modules alongside the on-chain support +functionality should be implemented in the Oasis SDK, including a convenient way +for ROFL to submit runtime transactions in a way that can be verified on-chain +as coming from a specific node/runtime instance. + +## Consequences + +### Positive + +- Oasis runtimes can easily be extended with arbitrary off-chain logic that can + securely interact with on-chain functionality. + +- Node operators do not need to perform much additional configuration in order + to support the new off-chain logic. + +### Negative + +- Additional complexity is introduced to the Runtime Host Protocol and to the + node binary. + +### Neutral + +## References diff --git a/README.md b/README.md index bc693d6..84274d2 100644 --- a/README.md +++ b/README.md @@ -115,4 +115,5 @@ The following records currently exist: * [ADR 0020](0020-governance-delegator-votes.md) - Governance Support for Delegator Votes * [ADR 0021](0021-keymanager-ephemeral-secrets.md) - Forward-Secret Ephemeral Secrets * [ADR 0022](0022-keymanager-master-secrets.md) - Forward-Secret Master Secrets +* [ADR 0024](0024-off-chain-runtime-logic.md) - Runtime Off-chain Logic (ROFL)