-
Notifications
You must be signed in to change notification settings - Fork 88
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
tools(feat): Add defined exceptions (#384)
* tools(feat): Add exception type * tests(fix): Use exception type * docs: Add exception tests description * docs(fix): tox fixes * docs(fix): more tox fixes * docs: Typed exceptions to consuming tests section * docs: remove link * changelog * feat(docs): add links to test examples using each exception type * Update docs/writing_tests/exception_tests.md Co-authored-by: danceratopz <[email protected]> --------- Co-authored-by: danceratopz <[email protected]>
- Loading branch information
1 parent
e2b84cc
commit 10d045f
Showing
29 changed files
with
450 additions
and
92 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Exceptions | ||
|
|
||
| Exception types are represented as a JSON string in the test fixtures. | ||
|
|
||
| The exception converted into a string is composed of the exception type name, | ||
| followed by a period, followed by the specific exception name. | ||
|
|
||
| For example, the exception `INSUFFICIENT_ACCOUNT_FUNDS` of type | ||
| `TransactionException` is represented as | ||
| `"TransactionException.INSUFFICIENT_ACCOUNT_FUNDS"`. | ||
|
|
||
| The JSON string can contain multiple exception types, separated by the `|` | ||
| character, denoting that the transaction or block can throw either one of | ||
| the exceptions. | ||
|
|
||
| ## `TransactionException` | ||
|
|
||
| ::: ethereum_test_tools.TransactionException | ||
|
|
||
| ## `BlockException` | ||
|
|
||
| ::: ethereum_test_tools.BlockException |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # Exception Tests | ||
|
|
||
| Exception tests are a special type of test which verify that an invalid transaction or an invalid block are correctly rejected with the expected error. | ||
|
|
||
| ## Creating an Exception Test | ||
|
|
||
| To test for an exception, the test can use either of the following types from `ethereum_test_tools` library: | ||
|
|
||
| 1. [`TransactionException`](../consuming_tests/exceptions.md#transactionexception): To be added to the `error` field of the `Transaction` object, and to the `exception` field of the `Block` object that includes the transaction; this exception type is used when a transaction is invalid, and therefore when included in a block, the block is expected to be invalid too. This is different from valid transactions where an exception during EVM execution is expected (e.g. a revert, or out-of-gas), which can be included in valid blocks. | ||
|
|
||
| For an example, see [`eip3860_initcode.test_initcode.test_contract_creating_tx`](../tests/shanghai/eip3860_initcode/test_initcode/index.md#tests.shanghai.eip3860_initcode.test_initcode.test_contract_creating_tx) which raises `TransactionException.INITCODE_SIZE_EXCEEDED` in the case that the initcode size exceeds the maximum allowed size. | ||
|
|
||
| 2. [`BlockException`](../consuming_tests/exceptions.md#blockexception): To be added to the `exception` field of the `Block` object; this exception type is used when a block is expected to be invalid, but the exception is related to a block property, e.g. an invalid value of the block header. | ||
|
|
||
| For an example, see [`eip4844_blobs.test_excess_blob_gas.test_invalid_static_excess_blob_gas`](../tests/cancun/eip4844_blobs/test_excess_blob_gas/index.md#tests.cancun.eip4844_blobs.test_excess_blob_gas.test_invalid_static_excess_blob_gas) which raises `BlockException.INCORRECT_EXCESS_BLOB_GAS` in the case that the the `excessBlobGas` remains unchanged | ||
| but the parent blobs included are not `TARGET_BLOBS_PER_BLOCK`. | ||
|
|
||
| Although exceptions can be combined with the `|` operator to indicate that a test vector can throw either one of multiple exceptions, ideally the tester should aim to use only one exception per test vector, and only use multiple exceptions on the rare instance when it is not possible to know which exception will be thrown because it depends on client implementation. | ||
|
|
||
| ## Adding a new exception | ||
|
|
||
| If a test requires a new exception, because none of the existing ones is suitable for the test, a new exception can be added to either [`TransactionException`](../consuming_tests/exceptions.md#transactionexception) or [`BlockException`](../consuming_tests/exceptions.md#blockexception) classes. | ||
|
|
||
| The new exception should be added as a new enum value, and the docstring of the attribute should be a string that describes the exception. | ||
|
|
||
| The name of the exception should be unique, and should not be used by any other exception. | ||
|
|
||
| ## Test runner behavior on exception tests | ||
|
|
||
| When an exception is added to a test vector, the test runner must check that the transaction or block is rejected with the expected exception. | ||
|
|
||
| The test runner must map the exception key to the corresponding error string that is expected to be returned by the client. | ||
|
|
||
| Exception mapping are particularly important in blockchain tests because the block can be invalid for multiple reasons, and the client returning a different error can mean that a verification in the client is faulty. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| """ | ||
| Exceptions for invalid execution. | ||
| """ | ||
|
|
||
| from .exceptions import BlockException, ExceptionList, ExceptionType, TransactionException | ||
|
|
||
| __all__ = ["BlockException", "ExceptionType", "ExceptionList", "TransactionException"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| """ | ||
| Exceptions for invalid execution. | ||
| """ | ||
|
|
||
| from enum import Enum, auto, unique | ||
| from typing import List, Union | ||
|
|
||
|
|
||
| class ExceptionList(list): | ||
| """ | ||
| A list of exceptions. | ||
| """ | ||
|
|
||
| def __init__(self, *exceptions: "ExceptionBase") -> None: | ||
| """ | ||
| Create a new ExceptionList. | ||
| """ | ||
| exceptions_set: List[ExceptionBase] = [] | ||
| for exception in exceptions: | ||
| if not isinstance(exception, ExceptionBase): | ||
| raise TypeError(f"Expected ExceptionBase, got {type(exception)}") | ||
| if exception not in exceptions_set: | ||
| exceptions_set.append(exception) | ||
| super().__init__(exceptions_set) | ||
|
|
||
| def __or__(self, other: Union["ExceptionBase", "ExceptionList"]) -> "ExceptionList": | ||
| """ | ||
| Combine two ExceptionLists. | ||
| """ | ||
| if isinstance(other, list): | ||
| return ExceptionList(*(self + other)) | ||
| return ExceptionList(*(self + [other])) | ||
|
|
||
| def __str__(self) -> str: | ||
| """ | ||
| String representation of the ExceptionList. | ||
| """ | ||
| return "|".join(str(exception) for exception in self) | ||
|
|
||
|
|
||
| class ExceptionBase(Enum): | ||
| """ | ||
| Base class for exceptions. | ||
| """ | ||
|
|
||
| def __or__( | ||
| self, | ||
| other: Union["TransactionException", "BlockException", ExceptionList], | ||
| ) -> "ExceptionList": | ||
| """ | ||
| Combine two exceptions into an ExceptionList. | ||
| """ | ||
| if isinstance(other, ExceptionList): | ||
| return ExceptionList(self, *other) | ||
| return ExceptionList(self, other) | ||
|
|
||
|
|
||
| @unique | ||
| class TransactionException(ExceptionBase): | ||
| """ | ||
| Exception raised when a transaction is invalid, and thus cannot be executed. | ||
| If a transaction with any of these exceptions is included in a block, the block is invalid. | ||
| """ | ||
|
|
||
| INSUFFICIENT_ACCOUNT_FUNDS = auto() | ||
| """ | ||
| Transaction's sender does not have enough funds to pay for the transaction. | ||
| """ | ||
| INSUFFICIENT_MAX_FEE_PER_GAS = auto() | ||
| """ | ||
| Transaction's max-fee-per-gas is lower than the block base-fee. | ||
| """ | ||
| PRIORITY_GREATER_THAN_MAX_FEE_PER_GAS = auto() | ||
| """ | ||
| Transaction's max-priority-fee-per-gas is greater than the max-fee-per-gas. | ||
| """ | ||
| INSUFFICIENT_MAX_FEE_PER_BLOB_GAS = auto() | ||
| """ | ||
| Transaction's max-fee-per-blob-gas is lower than the block's blob-gas price. | ||
| """ | ||
| INTRINSIC_GAS_TOO_LOW = auto() | ||
| """ | ||
| Transaction's gas limit is too low. | ||
| """ | ||
| INITCODE_SIZE_EXCEEDED = auto() | ||
| """ | ||
| Transaction's initcode for a contract-creating transaction is too large. | ||
| """ | ||
| TYPE_3_TX_PRE_FORK = auto() | ||
| """ | ||
| Transaction type 3 included before activation fork. | ||
| """ | ||
| TYPE_3_TX_ZERO_BLOBS_PRE_FORK = auto() | ||
| """ | ||
| Transaction type 3, with zero blobs, included before activation fork. | ||
| """ | ||
| TYPE_3_TX_INVALID_BLOB_VERSIONED_HASH = auto() | ||
| """ | ||
| Transaction contains a blob versioned hash with an invalid version. | ||
| """ | ||
| TYPE_3_TX_WITH_FULL_BLOBS = auto() | ||
| """ | ||
| Transaction contains full blobs (network-version of the transaction). | ||
| """ | ||
| TYPE_3_TX_BLOB_COUNT_EXCEEDED = auto() | ||
| """ | ||
| Transaction contains too many blob versioned hashes. | ||
| """ | ||
| TYPE_3_TX_CONTRACT_CREATION = auto() | ||
| """ | ||
| Transaction is a type 3 transaction and has an empty `to`. | ||
| """ | ||
| TYPE_3_TX_MAX_BLOB_GAS_ALLOWANCE_EXCEEDED = auto() | ||
| """ | ||
| Transaction causes block to go over blob gas limit. | ||
| """ | ||
| TYPE_3_TX_ZERO_BLOBS = auto() | ||
| """ | ||
| Transaction is type 3, but has no blobs. | ||
| """ | ||
|
|
||
|
|
||
| @unique | ||
| class BlockException(ExceptionBase): | ||
| """ | ||
| Exception raised when a block is invalid, but not due to a transaction. | ||
| E.g. all transactions in the block are valid, and can be applied to the state, but the | ||
| block header contains an invalid field. | ||
| """ | ||
|
|
||
| INCORRECT_BLOCK_FORMAT = auto() | ||
| """ | ||
| Block's format is incorrect, contains invalid fields, is missing fields, or contains fields of | ||
| a fork that is not active yet. | ||
| """ | ||
| BLOB_GAS_USED_ABOVE_LIMIT = auto() | ||
| """ | ||
| Block's blob gas used in header is above the limit. | ||
| """ | ||
| INCORRECT_BLOB_GAS_USED = auto() | ||
| """ | ||
| Block's blob gas used in header is incorrect. | ||
| """ | ||
| INCORRECT_EXCESS_BLOB_GAS = auto() | ||
| """ | ||
| Block's excess blob gas in header is incorrect. | ||
| """ | ||
|
|
||
|
|
||
| ExceptionType = Union[TransactionException, BlockException, ExceptionList] |
Oops, something went wrong.