refactor: Unified zeth utility #72
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hashcashier
added a commit
that referenced
this pull request
Feb 12, 2024
* transactions module * TxEssence trait * Generic Transaction struct * tx essence as a generic parameter * OptimismTxEssence * op chain spec * OpTxExecStrategy * redundant block builder type params * strategy bundles * host binary parameters * profiling flag * optimism * new derivation binary skeleton, copy over libs * providers and conversion utils * epoch transitioning * basic derive flow * host-side derivation * read metadata from op head * op-derive guest * disable guest memory leaks * heapless batch derivation * Add missing import * Remove heapless BinaryHeap * Remove heapless * Introduce op-derive tool * Remove ethers types from BatcherDb trait * Verify new op block has correct transaction list * Move derive logic into library * Fix bug in transaction trie reconstruction * Introduce get_op_header() to BatcherDb * Clippy warning * Default Serde value for FileProvider::receipts * Disable bloom filter checks * Clippy * Clippy * Fix parsing of from and to fields for deposits * Reintroduce filtering by log bloom * fmt * Add support for local exec to op-derive * Enforce block_number is correct in MemDb * Split derive() into multiple functions * Remove redundant check for batch parent hash * Remove redundant copy of system config * Remove redundant block number check * Cleanup * Remove redundant vector of eth blocks * Add base_fee_per_gas to Epoch * Store deposits in Epoch * Add Eth tail to DeriveOutput * Cleanup * Move deque_next_epoch_if_none to State * Move eth block processing to Batches * update zkvm, basic guest * more compose guest code, comment bonsai code * untested composition draft guest code * Reorg and cleanup * More cleanup * run cargo fmt --all * fix clippy warnings * remove unused imports * basic prep/left/fin in-memory flow * in memory aggregation workflow * format guest code * composition with receipts * add op-derive cmd test * add cmd tests * cleanup optimism/mod * Rename command line args * Fix test arguments * Rename config field to max_channel_bank_size * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity * Import from std instead of alloc/core * Re-enable core::mem::forget() optimization * Replace asserts with ensures; enforce absence of receipts for Op blocks * Simplfy iteration through derived transactions * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling * More println * Rework Batcher initialization * More logging * More log output if Bonsai workflow fails * refactor rpc db * variable derive step support * re-enable profiling * ignore rpc_cache dir * add -profile support to compose binary * upgrade zkvm, modify code comments * bump zkvm, fix CI * clippies * fixes and changes * vs code change * disambiguate merkle ranges and proofs Co-authored-by: Wolfgang Welz <[email protected]> * update risc0 to release v0.20 * update GH action to 0.20 * use old actions * changes * refactor: Unified zeth utility (#72) * feat: Provably build derived op blocks (#78) * feat: Composition via Bonsai (#79) * feat: Deterministic builds with receipt cache (+misc) (#80) * cleanup toml files * remove unused imports * cli cleanup * change block count to u32 * test composition * fix arguments * update risc0 to 0.20.1 * compact json files * update lock files * test for warnings * nits * docker in readme * create cache file if not found * update copyright * update copyright --------- Co-authored-by: Timothy Carstens <[email protected]> Co-authored-by: Wolfgang Welz <[email protected]>
hashcashier
added a commit
that referenced
this pull request
Feb 12, 2024
* transactions module * TxEssence trait * Generic Transaction struct * tx essence as a generic parameter * OptimismTxEssence * op chain spec * OpTxExecStrategy * redundant block builder type params * strategy bundles * host binary parameters * profiling flag * optimism * new derivation binary skeleton, copy over libs * providers and conversion utils * epoch transitioning * basic derive flow * host-side derivation * read metadata from op head * op-derive guest * disable guest memory leaks * heapless batch derivation * Add missing import * Remove heapless BinaryHeap * Remove heapless * Introduce op-derive tool * Remove ethers types from BatcherDb trait * Verify new op block has correct transaction list * Move derive logic into library * Fix bug in transaction trie reconstruction * Introduce get_op_header() to BatcherDb * Clippy warning * Default Serde value for FileProvider::receipts * Disable bloom filter checks * Clippy * Clippy * Fix parsing of from and to fields for deposits * Reintroduce filtering by log bloom * fmt * Add support for local exec to op-derive * Enforce block_number is correct in MemDb * Split derive() into multiple functions * Remove redundant check for batch parent hash * Remove redundant copy of system config * Remove redundant block number check * Cleanup * Remove redundant vector of eth blocks * Add base_fee_per_gas to Epoch * Store deposits in Epoch * Add Eth tail to DeriveOutput * Cleanup * Move deque_next_epoch_if_none to State * Move eth block processing to Batches * update zkvm, basic guest * more compose guest code, comment bonsai code * untested composition draft guest code * Reorg and cleanup * More cleanup * run cargo fmt --all * fix clippy warnings * remove unused imports * basic prep/left/fin in-memory flow * in memory aggregation workflow * format guest code * composition with receipts * add op-derive cmd test * add cmd tests * cleanup optimism/mod * Rename command line args * Fix test arguments * Rename config field to max_channel_bank_size * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity * Import from std instead of alloc/core * Re-enable core::mem::forget() optimization * Replace asserts with ensures; enforce absence of receipts for Op blocks * Simplfy iteration through derived transactions * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling * More println * Rework Batcher initialization * More logging * More log output if Bonsai workflow fails * refactor rpc db * variable derive step support * re-enable profiling * ignore rpc_cache dir * add -profile support to compose binary * upgrade zkvm, modify code comments * initial monolith * refactor prompt style * op-info support * delete old host binaries * nits * fix tests * help nit * short params * bump zkvm, fix CI * clippies * save receipts * refactor * move clap structs to cli module * refactor * refactor main monolith * fixes and changes * vs code change * default cache folder nit * build after derive * disambiguate merkle ranges and proofs Co-authored-by: Wolfgang Welz <[email protected]> * missing cache files for test * update crates * prove on bonsai * verify bonsai receipts * readme,cache,cuda,no-dev-mode,receipt uuid output, etc.. * logging nit * rlp decoding transactions * provably bad blocks * build on derive * derivation test fix * support erroneous preflights * host: derive with receipts * fix op head * verification nit * repeat bonsai requests until success * wider bonsai loop * switch to async bonsai * update readme * patch blocking code * patch blocking code * rpc_cache -> cache_rpc * det build, receipt cache, info->debug * fix nondet receipt label * logging and TransactionKind rlp decode fix * update risc0 to release v0.20 * cargo locks * nit * move more info output to trace * update GH action to 0.20 * use old actions * stark2snark * changes * refactor rlp & comments * update to v20.1 * refactor: Unified zeth utility (#72) * feat: Provably build derived op blocks (#78) * feat: Composition via Bonsai (#79) * feat: Deterministic builds with receipt cache (+misc) (#80) * cleanup toml files * remove unused imports * fmt * SNARK Verification via RPC call * cli cleanup * change block count to u32 * test composition * fix arguments * update risc0 to 0.20.1 * compact json files * update lock files * test for warnings * nits * docker in readme * remove env unset code because #91 --------- Co-authored-by: Timothy Carstens <[email protected]> Co-authored-by: Wolfgang Welz <[email protected]>
hashcashier
added a commit
that referenced
this pull request
Feb 12, 2024
* transactions module * TxEssence trait * Generic Transaction struct * tx essence as a generic parameter * OptimismTxEssence * op chain spec * OpTxExecStrategy * redundant block builder type params * strategy bundles * host binary parameters * profiling flag * optimism * new derivation binary skeleton, copy over libs * providers and conversion utils * epoch transitioning * basic derive flow * host-side derivation * read metadata from op head * op-derive guest * disable guest memory leaks * heapless batch derivation * Add missing import * Remove heapless BinaryHeap * Remove heapless * Introduce op-derive tool * Remove ethers types from BatcherDb trait * Verify new op block has correct transaction list * Move derive logic into library * Fix bug in transaction trie reconstruction * Introduce get_op_header() to BatcherDb * Clippy warning * Default Serde value for FileProvider::receipts * Disable bloom filter checks * Clippy * Clippy * Fix parsing of from and to fields for deposits * Reintroduce filtering by log bloom * fmt * Add support for local exec to op-derive * Enforce block_number is correct in MemDb * Split derive() into multiple functions * Remove redundant check for batch parent hash * Remove redundant copy of system config * Remove redundant block number check * Cleanup * Remove redundant vector of eth blocks * Add base_fee_per_gas to Epoch * Store deposits in Epoch * Add Eth tail to DeriveOutput * Cleanup * Move deque_next_epoch_if_none to State * Move eth block processing to Batches * update zkvm, basic guest * more compose guest code, comment bonsai code * untested composition draft guest code * Reorg and cleanup * More cleanup * run cargo fmt --all * fix clippy warnings * remove unused imports * basic prep/left/fin in-memory flow * in memory aggregation workflow * format guest code * composition with receipts * add op-derive cmd test * add cmd tests * cleanup optimism/mod * Rename command line args * Fix test arguments * Rename config field to max_channel_bank_size * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity * Import from std instead of alloc/core * Re-enable core::mem::forget() optimization * Replace asserts with ensures; enforce absence of receipts for Op blocks * Simplfy iteration through derived transactions * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling * More println * Rework Batcher initialization * More logging * More log output if Bonsai workflow fails * refactor rpc db * variable derive step support * re-enable profiling * ignore rpc_cache dir * add -profile support to compose binary * upgrade zkvm, modify code comments * bump zkvm, fix CI * clippies * fixes and changes * vs code change * disambiguate merkle ranges and proofs Co-authored-by: Wolfgang Welz <[email protected]> * update risc0 to release v0.20 * update GH action to 0.20 * use old actions * changes * refactor: Unified zeth utility (#72) * feat: Provably build derived op blocks (#78) * feat: Composition via Bonsai (#79) * feat: Deterministic builds with receipt cache (+misc) (#80) * cleanup toml files * remove unused imports * cli cleanup * change block count to u32 * test composition * fix arguments * update risc0 to 0.20.1 * compact json files * update lock files * deterministic cache * test for warnings * nits * docker in readme * create cache file if not found * fmt * update copyright * update copyright --------- Co-authored-by: Rami Khalil <[email protected]> Co-authored-by: Timothy Carstens <[email protected]> Co-authored-by: Rami <[email protected]>
hashcashier
added a commit
that referenced
this pull request
Feb 13, 2024
* transactions module * TxEssence trait * Generic Transaction struct * tx essence as a generic parameter * OptimismTxEssence * op chain spec * OpTxExecStrategy * redundant block builder type params * strategy bundles * host binary parameters * profiling flag * optimism * new derivation binary skeleton, copy over libs * providers and conversion utils * epoch transitioning * basic derive flow * host-side derivation * read metadata from op head * op-derive guest * disable guest memory leaks * heapless batch derivation * Add missing import * Remove heapless BinaryHeap * Remove heapless * Introduce op-derive tool * Remove ethers types from BatcherDb trait * Verify new op block has correct transaction list * Move derive logic into library * Fix bug in transaction trie reconstruction * Introduce get_op_header() to BatcherDb * Clippy warning * Default Serde value for FileProvider::receipts * Disable bloom filter checks * Clippy * Clippy * Fix parsing of from and to fields for deposits * Reintroduce filtering by log bloom * fmt * Add support for local exec to op-derive * Enforce block_number is correct in MemDb * Split derive() into multiple functions * Remove redundant check for batch parent hash * Remove redundant copy of system config * Remove redundant block number check * Cleanup * Remove redundant vector of eth blocks * Add base_fee_per_gas to Epoch * Store deposits in Epoch * Add Eth tail to DeriveOutput * Cleanup * Move deque_next_epoch_if_none to State * Move eth block processing to Batches * update zkvm, basic guest * more compose guest code, comment bonsai code * untested composition draft guest code * Reorg and cleanup * More cleanup * run cargo fmt --all * fix clippy warnings * remove unused imports * basic prep/left/fin in-memory flow * in memory aggregation workflow * format guest code * composition with receipts * add op-derive cmd test * add cmd tests * cleanup optimism/mod * Rename command line args * Fix test arguments * Rename config field to max_channel_bank_size * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity * Import from std instead of alloc/core * Re-enable core::mem::forget() optimization * Replace asserts with ensures; enforce absence of receipts for Op blocks * Simplfy iteration through derived transactions * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling * More println * Rework Batcher initialization * More logging * More log output if Bonsai workflow fails * refactor rpc db * variable derive step support * re-enable profiling * ignore rpc_cache dir * add -profile support to compose binary * upgrade zkvm, modify code comments * bump zkvm, fix CI * clippies * fixes and changes * vs code change * disambiguate merkle ranges and proofs Co-authored-by: Wolfgang Welz <[email protected]> * update risc0 to release v0.20 * update GH action to 0.20 * use old actions * changes * refactor: Unified zeth utility (#72) * feat: Provably build derived op blocks (#78) * feat: Composition via Bonsai (#79) * feat: Deterministic builds with receipt cache (+misc) (#80) * cleanup toml files * remove unused imports * cli cleanup * change block count to u32 * test composition * fix arguments * update risc0 to 0.20.1 * compact json files * update lock files * improve execution * test for warnings * nits * docker in readme * create cache file if not found * update copyright * update copyright --------- Co-authored-by: Rami Khalil <[email protected]> Co-authored-by: Timothy Carstens <[email protected]> Co-authored-by: Rami <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR integrates all features into a single CLI binary with a command based usage pattern. The task to be performed by the binary (preflight/exec/prove/verify) is now explicit in the command called rather than implied by the provided arguments.
For every command, the
--networkparameter can be set to eitherethereumoroptimismfor provable derivation of single blocks from either chain. This provides the basic functionality of the originalmainbinary.With
--network=optimism-derived, the optimism blocks requested are derived using the data available on the ethereum chain. The number of sequential op blocks to derive is set through the--block-countparameter. The derivation proof creation method is done without proof composition by default, requiring the derivation to fit inside a single zkVM execution.The derivation proof can alternatively be created using proof composition by setting the
--compositionparameter to the number of op blocks per rolled up proof. However, theruncommand does not support this parameter because receipts are required for the proof composition. Consequently, one has to call theprovecommand in dev mode for the same functionality.To verify existing receipts:
Lastly, the
op-infobinary has also been integrated as a stand alone command: