From 98c96ef2987b8c2d27758b0abd18c83e09bb5408 Mon Sep 17 00:00:00 2001 From: Lucas Molas Date: Mon, 12 Oct 2020 12:45:18 -0300 Subject: [PATCH 1/2] rename BlockSync to ChainExchange --- content/algorithms/block_sync.md | 101 ----------------------- content/algorithms/chain_exchange.md | 116 +++++++++++++++++++++++++++ 2 files changed, 116 insertions(+), 101 deletions(-) delete mode 100644 content/algorithms/block_sync.md create mode 100644 content/algorithms/chain_exchange.md diff --git a/content/algorithms/block_sync.md b/content/algorithms/block_sync.md deleted file mode 100644 index 32a5f0172..000000000 --- a/content/algorithms/block_sync.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -title: "BlockSync" -weight: 5 -dashboardWeight: 1 -dashboardState: stable -dashboardAudit: missing -dashboardTests: 0 ---- - -# BlockSync - -{{< hint info >}} -**Name**: Block Sync -**Protocol ID**: `/fil/sync/blk/0.0.1` -{{< /hint >}} - -BlockSync is a simple request/response protocol that allows Filecoin nodes to request ranges of Tipsets from each other, when they have run out of sync, e.g., during downtime. Given that the Filecoin blockchain is extended in Tipsets (i.e., groups of blocks), rather than in blocks, the BlockSync protocol operates in terms of Tipsets. - -The request message requests a chain segment of a given length by the hash of its highest Tipset. It is worth noting that this does not necessarily apply to the head (i.e., latest Tipset) of the current chain only, but it can also apply to earlier segments. For example, if the current height is at 5000, but a node is missing Tipsets between 4500-4700, then the `Head` requested is 4700 and the length is 200. - -The `Options` allow the requester to specify whether they want to receive block headers of the Tipsets only, the transaction messages included in every block, or both. - -The response contains the requested chain segment in reverse iteration order. Each item in the `Chain` array contains either the block headers for that Tipset if the `Blocks` option bit in the request is set, or the messages across all blocks in that Tipset, if the `Messages` bit is set, or both, if both option bits are set. - -The `MsgIncludes` array contains one array of integers for each block in the `Blocks` array. Each of the `Blocks` arrays in `MsgIncludes` contains a list of message indexes from the `Messages` array that are in each `Block` in the blocks array. - -If not all Tipsets requested could be fetched, but the `Head` of the chain segment requested was successfully fetched, then this is not considered an error, given that the node can continue extending the chain from the `Head` onwards. - -```go -type BlockSyncRequest struct { - ## The TipSet being synced from - start [&Block] - ## How many tipsets to sync - requestLength UInt - ## Query options - options Options -} -``` - -```go -type Options enum { - # Include only blocks - | Blocks 1 - # Include only messages - | Messages 2 - # Include messages and blocks - | BlocksAndMessages 3 -} - -type BlockSyncResponse struct { - chain [TipSetBundle] - status Status -} - -type TipSetBundle struct { - blocks [Blocks] - - blsMsgs [Message] - blsMsgIncludes [[Uint]] - - secpMsgs [SignedMessage] - secpMsgIncludes [[UInt]] -} - -type Status enum { - ## All is well. - | Success 0 - ## We could not fetch all blocks requested (but at least we returned - ## the `Head` requested). Not considered an error. - | PartialResponse 101 - ## Request.Start not found. - | BlockNotFound 201 - ## Requester is making too many requests. - | GoAway 202 - ## Internal error occured. - | InternalError 203 - ## Request was bad - | BadRequest 204 -} -``` - -## Example - -For the set of arrays in the following TipSetBundle, the corresponding messages per block are as shown below: - -**TipSetBundle** -```js -Blocks: [b0, b1] -secpMsgs: [mA, mB, mC, mD] -secpMsgIncludes: [[0, 1, 3], [1, 2, 0]] -``` - -**Messages corresponding to each Block** -```js -Block 'b0': [mA, mB, mD] -Block 'b1': [mB, mC, mA] -``` - -In other words, the first element of the `secpMsgIncludes` array: `[0, 1, 3]` points to the 1st, 2nd and 4th element of the `secpMsgs` array: `mA, mB, mD`, which correspond to the 1st element of the `Blocks` array `b0`. Hence, `Block 'b0': [mA, mB, mD]`. - -Similarly, the second element of the `secpMsgIncludes` array: `[1, 2, 0]` points to the 2nd, 3rd and 1st element of the `secpMsgs` array: `mB, mC, mA`, which correspond to the 2nd element of the `Blocks` array `b1`. Hence, `Block 'b1': [mB, mC, mA]`. \ No newline at end of file diff --git a/content/algorithms/chain_exchange.md b/content/algorithms/chain_exchange.md new file mode 100644 index 000000000..2a476d41e --- /dev/null +++ b/content/algorithms/chain_exchange.md @@ -0,0 +1,116 @@ +--- +title: "ChainExchange" +weight: 5 +dashboardWeight: 1 +dashboardState: stable +dashboardAudit: missing +dashboardTests: 0 +--- + +# ChainExchange + +{{< hint info >}} +**Name**: Chain Exchange +**Protocol ID**: `/fil/chain/xchg/0.0.1` +{{< /hint >}} + +ChainExchange is a simple request/response protocol that allows Filecoin nodes to request ranges of Tipsets and/or Messages from each other. + +The `Request` message requests a chain segment of a given length by the hash of the highest Tipset in the segment (not necessarily heaviest Tipset of the current chain). For example, if the current height is at 5000, but a node is missing Tipsets between 4500-4700, then the `Request.Head` requested is 4700 and `Request.Length` is 200. + +The `Options` allow the requester to specify whether they want to receive block headers of the Tipsets only, the transaction messages included in every block, or both. + +The `Response` contains the requested chain segment in reverse iteration order. Each item in the `Chain` array contains either the block headers for that Tipset if the `Blocks` option bit in the request is set, or the messages across all blocks in that Tipset, if the `Messages` bit is set, or both, if both option bits are set. + +Each `CompactedMessages` structure contains the BLS and `secp256k1` messages for the corresponding Tipset in unified arrays that encode each message once regardless of how many times it appears in the Tipset's blocks (to reduce the `Response` size). The mapping between message and the blocks into which they are included is encoded in the `BlsIncludes` and `SecpkIncludes` arrays. These arrays are indexed by the block index in the Tipset and contain in that position an array of message indexes that belong to that block. The messages themselves are in the `Bls` and `Secpk` arrays. + +If not all Tipsets requested could be fetched, but the `Head` of the chain segment requested was successfully fetched (and potentially more contiguous Tipsets), then this is not considered an error and a `Partial` response code is returned. The node can continue extending the chain from the partial returned segment onwards. + +```go +type Request struct { + ## Head of the requested segment (block CIDs comprising the entire Tipset) + Head [Cid] + ## Length of the requested segment + Length UInt + ## Query options + Options UInt +} +``` + +```go +type Options enum { + # Include block headers + | Headers 1 + # Include block messages + | Messages 2 +} + +type Response struct { + ## Response Status + Status status + ## Optional error message + ErrorMessage string + ## Returned segment containing block messages and/or headers + Chain []*BSTipSet +} + +type Status enum { + ## Success: the entire segment requested was fetched. + | Ok 0 + ## We could not fetch all Tipsets requested but at least we returned + ## the `Head` requested (and potentially more contiguous Tipsets). + ## Not considered an error. + | Partial 101 + ## `Request.Head` not found. + | NotFound 201 + ## Requester is making too many requests. + | GoAway 202 + ## Internal error occurred. + | InternalError 203 + ## Request was badly formed. + | BadRequest 204 +} + +type CompactedMessages struct { + ## Array of BLS messages in this tipset. + Bls [Message] + ## Array of messages indexes present in each block: + ## `BlsIncludes[BI] -> [MI]` + ## * BI: block index in the tipset. + ## * MI: message index in `Bls` list. + BlsIncludes [[Uint]] + + ## Array of `secp256k1` messages. + Secpk [SignedMessage] + ## Inclusion array, see `BlsIncludes`. + SecpkIncludes [[UInt]] +} +``` + +## Example + +For the set of arrays in the following `BSTipSet`, the corresponding messages per block are as shown below: + +**BSTipSet** +```js +Blocks: [b0, b1] +CompactedMessages +``` + +**CompactedMessages** +```js +Secpk: [mA, mB, mC, mD] +SecpkIncludes: [[0, 1, 3], [1, 2, 0]] +``` + +**Messages corresponding to each Block** +```js +Block 'b0': [mA, mB, mD] +Block 'b1': [mB, mC, mA] +``` + +In other words, the first element of the `SecpkIncludes` array: `[0, 1, 3]` points to the 1st, 2nd and 4th element of the `Secpk` array: `mA, mB, mD`, which correspond to the 1st element of the `Blocks` array `b0`. Hence, `Block 'b0': [mA, mB, mD]`. + +Similarly, the second element of the `SecpkIncludes` array: `[1, 2, 0]` points to the 2nd, 3rd and 1st element of the `Secpk` array: `mB, mC, mA`, which correspond to the 2nd element of the `Blocks` array `b1`. Hence, `Block 'b1': [mB, mC, mA]`. + +See [Lotus](https://github.com/filecoin-project/lotus/tree/master/chain/exchange) for an example implementation of the protocol. From d1bd8ef8c481a8bc404a640e274cfa97634ceaf3 Mon Sep 17 00:00:00 2001 From: Lucas Molas Date: Wed, 14 Oct 2020 16:02:29 -0300 Subject: [PATCH 2/2] Update content/algorithms/chain_exchange.md Co-authored-by: Yiannis Psaras <52073247+yiannisbot@users.noreply.github.com> --- content/algorithms/chain_exchange.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/algorithms/chain_exchange.md b/content/algorithms/chain_exchange.md index 2a476d41e..f388aa7d6 100644 --- a/content/algorithms/chain_exchange.md +++ b/content/algorithms/chain_exchange.md @@ -14,7 +14,7 @@ dashboardTests: 0 **Protocol ID**: `/fil/chain/xchg/0.0.1` {{< /hint >}} -ChainExchange is a simple request/response protocol that allows Filecoin nodes to request ranges of Tipsets and/or Messages from each other. +ChainExchange is a simple request/response protocol that allows Filecoin nodes to request ranges of Tipsets and/or Messages from each other. `ChainExchange` is part of a bigger "family" of protocols that live under the umbrella of `ChainSync`. Please refer to the [ChainSync](blockchain#chainsync) section for an overview of the operational boundaries of each of the protocols in ChainSync. The `Request` message requests a chain segment of a given length by the hash of the highest Tipset in the segment (not necessarily heaviest Tipset of the current chain). For example, if the current height is at 5000, but a node is missing Tipsets between 4500-4700, then the `Request.Head` requested is 4700 and `Request.Length` is 200.