Merge branch 'main' into opcode-parameter-count

docs/CHANGELOG.md

@@ -8,11 +8,14 @@ Test fixtures for use by clients are available for each release on the [Github r
 
 ### 🧪 Test Cases
 
+- ✨ Add `test_double_kill` and `test_recreate` which test resurrection of accounts killed with `SELFDESTRUCT` ([#488](https://github.com/ethereum/execution-spec-tests/pull/488)).
+
 ### 🛠️ Framework
 
 - 🐞 Fix incorrect `!=` operator for `FixedSizeBytes` ([#477](https://github.com/ethereum/execution-spec-tests/pull/477)).
 - ✨ Add Macro enum that represents byte sequence of Op instructions ([#457](https://github.com/ethereum/execution-spec-tests/pull/457))
 - ✨ Number of parameters used to call opcodes (to generate bytecode) is now checked ([#492](https://github.com/ethereum/execution-spec-tests/pull/492)).
+- ✨ Libraries have been refactored to use `pydantic` for type checking in most test types ([#486](https://github.com/ethereum/execution-spec-tests/pull/486)).
 
 ### 🔧 EVM Tools
 
@@ -21,6 +24,9 @@ Test fixtures for use by clients are available for each release on the [Github r
 - 🐞 Fix CI by using Golang 1.21 in Github Actions to build geth ([#484](https://github.com/ethereum/execution-spec-tests/pull/484)).
 - 💥 "Merge" has been renamed to "Paris" in the "network" field of the Blockchain tests, and in the "post" field of the State tests ([#480](https://github.com/ethereum/execution-spec-tests/pull/480)).
 - ✨ Port entry point scripts to use [click](https://click.palletsprojects.com) and add tests ([#483](https://github.com/ethereum/execution-spec-tests/pull/483)).
+- 💥 As part of the pydantic conversion, the fixtures have the following (possibly breaking) changes ([#486](https://github.com/ethereum/execution-spec-tests/pull/486)):
+  - State test field `transaction` now uses the proper zero-padded hex number format for fields `maxPriorityFeePerGas`, `maxFeePerGas`, and `maxFeePerBlobGas`
+  - Fixtures' hashes (in the `_info` field) are now calculated by removing the "_info" field entirely instead of it being set to an empty dict.
 
 ## 🔜 [v2.1.1](https://github.com/ethereum/execution-spec-tests/releases/tag/v2.1.1) - 2024-03-09
 

docs/writing_tests/code_standards.md

@@ -2,17 +2,17 @@
 
 The Python code in the tests subdirectory `./tests` must fulfill the following checks:
 
-|   | Command                 | Explanation                                                               |
-|---|-------------------------|---------------------------------------------------------------------------|
-| 1 | `fname8 tests`          | Spell check passes using the `./whitelist.txt` dictionary file.           |
-| 2 | `isort tests --check --diff` | Python imports ordered and arranged according to isort's standards.  |
-| 3 | `black tests --check --diff` | Python source must be black-formatted.                               |
-| 4 | `flake8 tests`          | Python lint and spell-checked.                                            |
-| 5 | `mypy tests`            | Objects that provide typehints pass type-checking via mypy.               |
-| 6 | `fill`                  | All tests tests must execute correctly.                                   |
-| 7 | `mkdocs build --strict` | Documentation generated without warnings.                                 |
-| 8 | `pyspelling`            | Markdown spell-check.                                                     |
-| 9 | `markdownlint-cli2`     | Markdown lint check.                                                      |
+|   | Command                 | Explanation                                                              |
+|---|-------------------------|--------------------------------------------------------------------------|
+| 1 | `fname8 tests`          | Spell check passes using the `./whitelist.txt` dictionary file.          |
+| 2 | `isort tests --check --diff` | Python imports ordered and arranged according to isort's standards. |
+| 3 | `black tests --check --diff` | Python source must be black-formatted.                              |
+| 4 | `flake8 tests`          | Python lint and spell-checked.                                           |
+| 5 | `mypy tests`            | Objects that provide typehints pass type-checking via mypy.              |
+| 6 | `fill`                  | All tests must execute correctly.                                        |
+| 7 | `mkdocs build --strict` | Documentation generated without warnings.                                |
+| 8 | `pyspelling`            | Markdown spell-check.                                                    |
+| 9 | `markdownlint-cli2`     | Markdown lint check.                                                     |
 
 While this seems like a long list, a correctly configured editor (see [VS Code Setup](../getting_started/setup_vs_code.md)) essentially assures:
 

docs/writing_tests/exception_tests.md

@@ -12,7 +12,7 @@ To test for an exception, the test can use either of the following types from `e
 
 2. [`BlockException`](../consuming_tests/exceptions.md#blockexception): To be added to the `exception` field of the `Block` object; this exception type is used when a block is expected to be invalid, but the exception is related to a block property, e.g. an invalid value of the block header.
 
-    For an example, see [`eip4844_blobs.test_excess_blob_gas.test_invalid_static_excess_blob_gas`](../tests/cancun/eip4844_blobs/test_excess_blob_gas/index.md#tests.cancun.eip4844_blobs.test_excess_blob_gas.test_invalid_static_excess_blob_gas) which raises `BlockException.INCORRECT_EXCESS_BLOB_GAS` in the case that the the `excessBlobGas` remains unchanged
+    For an example, see [`eip4844_blobs.test_excess_blob_gas.test_invalid_static_excess_blob_gas`](../tests/cancun/eip4844_blobs/test_excess_blob_gas/index.md#tests.cancun.eip4844_blobs.test_excess_blob_gas.test_invalid_static_excess_blob_gas) which raises `BlockException.INCORRECT_EXCESS_BLOB_GAS` in the case that the `excessBlobGas` remains unchanged
     but the parent blobs included are not `TARGET_BLOBS_PER_BLOCK`.
 
 Although exceptions can be combined with the `|` operator to indicate that a test vector can throw either one of multiple exceptions, ideally the tester should aim to use only one exception per test vector, and only use multiple exceptions on the rare instance when it is not possible to know which exception will be thrown because it depends on client implementation.

pyproject.toml

@@ -12,3 +12,4 @@ line-length = 99
 
 [tool.mypy]
 mypy_path = "$MYPY_CONFIG_FILE_DIR/stubs"
+plugins = ["pydantic.mypy"]

setup.cfg

@@ -34,6 +34,7 @@ install_requires =
     trie==2.1.1
     semver==3.0.1
     click>=8.0.0,<9
+    pydantic>=2.6.3
 
 [options.package_data]
 ethereum_test_tools =
@@ -61,7 +62,7 @@ test =
 
 lint =
     isort>=5.8,<6
-    mypy==0.982; implementation_name == "cpython"
+    mypy==0.991; implementation_name == "cpython"
     types-requests
     black==22.3.0; implementation_name == "cpython"
     flake8-spellcheck>=0.24,<0.25

src/ethereum_test_tools/__init__.py

@@ -18,11 +18,10 @@
     AccessList,
     Account,
     Address,
-    Auto,
+    Alloc,
     EngineAPIError,
     Environment,
     Hash,
-    JSONEncoder,
     Removable,
     Storage,
     TestAddress,
@@ -39,9 +38,8 @@
     copy_opcode_cost,
     cost_memory_bytes,
     eip_2028_transaction_data_cost,
-    transaction_list_root,
 )
-from .exceptions import BlockException, ExceptionList, ExceptionType, TransactionException
+from .exceptions import BlockException, TransactionException
 from .reference_spec import ReferenceSpec, ReferenceSpecTypes
 from .spec import (
     SPEC_TYPES,
@@ -62,7 +60,7 @@
     "AccessList",
     "Account",
     "Address",
-    "Auto",
+    "Alloc",
     "BaseFixture",
     "BaseTest",
     "Block",
@@ -76,13 +74,10 @@
     "Conditional",
     "EngineAPIError",
     "Environment",
-    "ExceptionList",
-    "ExceptionType",
     "FixtureCollector",
     "Hash",
     "Header",
     "Initcode",
-    "JSONEncoder",
     "Opcode",
     "Macro",
     "OpcodeCallArg",
@@ -113,5 +108,4 @@
     "cost_memory_bytes",
     "eip_2028_transaction_data_cost",
     "eip_2028_transaction_data_cost",
-    "transaction_list_root",
 )

src/ethereum_test_tools/common/__init__.py

@@ -1,6 +1,7 @@
 """
 Common definitions and types.
 """
+
 from .base_types import (
     Address,
     Bloom,
@@ -14,6 +15,7 @@
 from .constants import (
     AddrAA,
     AddrBB,
+    EmptyOmmersRoot,
     EmptyTrieRoot,
     EngineAPIError,
     TestAddress,
@@ -36,18 +38,11 @@
     AccessList,
     Account,
     Alloc,
-    Auto,
     Environment,
-    JSONEncoder,
     Removable,
     Storage,
     Transaction,
     Withdrawal,
-    alloc_to_accounts,
-    serialize_transactions,
-    str_or_none,
-    transaction_list_root,
-    withdrawals_root,
 )
 
 __all__ = (
@@ -57,16 +52,15 @@
     "AddrAA",
     "AddrBB",
     "Alloc",
-    "Auto",
     "Bloom",
     "Bytes",
     "EngineAPIError",
+    "EmptyOmmersRoot",
     "EmptyTrieRoot",
     "Environment",
     "Hash",
     "HeaderNonce",
     "HexNumber",
-    "JSONEncoder",
     "Number",
     "Removable",
     "Storage",
@@ -79,16 +73,11 @@
     "Withdrawal",
     "ZeroPaddedHexNumber",
     "add_kzg_version",
-    "alloc_to_accounts",
     "ceiling_division",
     "compute_create_address",
     "compute_create2_address",
     "copy_opcode_cost",
     "cost_memory_bytes",
     "eip_2028_transaction_data_cost",
-    "serialize_transactions",
-    "str_or_none",
     "to_json",
-    "transaction_list_root",
-    "withdrawals_root",
 )

src/ethereum_test_tools/common/base_types.py

@@ -2,7 +2,14 @@
 Basic type primitives used to define other types.
 """
 
-from typing import ClassVar, SupportsBytes, Type, TypeVar
+from typing import Any, ClassVar, SupportsBytes, Type, TypeVar
+
+from pydantic import GetCoreSchemaHandler
+from pydantic_core.core_schema import (
+    PlainValidatorFunctionSchema,
+    no_info_plain_validator_function,
+    to_string_ser_schema,
+)
 
 from .conversions import (
     BytesConvertible,
@@ -12,12 +19,29 @@
     to_fixed_size_bytes,
     to_number,
 )
-from .json import JSONEncoder, SupportsJSON
 
 N = TypeVar("N", bound="Number")
 
 
-class Number(int, SupportsJSON):
+class ToStringSchema:
+    """
+    Type converter to add a simple pydantic schema that correctly parses and serializes the type.
+    """
+
+    @staticmethod
+    def __get_pydantic_core_schema__(
+        source_type: Any, handler: GetCoreSchemaHandler
+    ) -> PlainValidatorFunctionSchema:
+        """
+        Calls the class constructor without info and appends the serialization schema.
+        """
+        return no_info_plain_validator_function(
+            source_type,
+            serialization=to_string_ser_schema(),
+        )
+
+
+class Number(int, ToStringSchema):
     """
     Class that helps represent numbers in tests.
     """
@@ -34,12 +58,6 @@ def __str__(self) -> str:
         """
         return str(int(self))
 
-    def __json__(self, encoder: JSONEncoder) -> str:
-        """
-        Returns the JSON representation of the number.
-        """
-        return str(self)
-
     def hex(self) -> str:
         """
         Returns the hexadecimal representation of the number.
@@ -85,7 +103,10 @@ def hex(self) -> str:
         return "0x" + hex_str
 
 
-class Bytes(bytes, SupportsJSON):
+NumberBoundTypeVar = TypeVar("NumberBoundTypeVar", Number, HexNumber, ZeroPaddedHexNumber)
+
+
+class Bytes(bytes, ToStringSchema):
     """
     Class that helps represent bytes of variable length in tests.
     """
@@ -108,12 +129,6 @@ def __str__(self) -> str:
         """
         return self.hex()
 
-    def __json__(self, encoder: JSONEncoder) -> str:
-        """
-        Returns the JSON representation of the bytes.
-        """
-        return str(self)
-
     def hex(self, *args, **kwargs) -> str:
         """
         Returns the hexadecimal representation of the bytes.
@@ -130,6 +145,71 @@ def or_none(cls, input: "Bytes | BytesConvertible | None") -> "Bytes | None":
         return cls(input)
 
 
+S = TypeVar("S", bound="FixedSizeHexNumber")
+
+
+class FixedSizeHexNumber(int, ToStringSchema):
+    """
+    A base class that helps represent an integer as a fixed byte-length
+    hexadecimal number.
+
+    This class is used to dynamically generate subclasses of a specific byte
+    length.
+    """
+
+    byte_length: ClassVar[int]
+    max_value: ClassVar[int]
+
+    def __class_getitem__(cls, length: int) -> Type["FixedSizeHexNumber"]:
+        """
+        Creates a new FixedSizeHexNumber class with the given length.
+        """
+
+        class Sized(cls):  # type: ignore
+            byte_length = length
+            max_value = 2 ** (8 * length) - 1
+
+        return Sized
+
+    def __new__(cls, input: NumberConvertible | N):
+        """
+        Creates a new Number object.
+        """
+        i = to_number(input)
+        if i > cls.max_value:
+            raise ValueError(f"Value {i} is too large for {cls.byte_length} bytes")
+        if i < 0:
+            i += cls.max_value + 1
+            if i <= 0:
+                raise ValueError(f"Value {i} is too small for {cls.byte_length} bytes")
+        return super(FixedSizeHexNumber, cls).__new__(cls, i)
+
+    def __str__(self) -> str:
+        """
+        Returns the string representation of the number.
+        """
+        return self.hex()
+
+    def hex(self) -> str:
+        """
+        Returns the hexadecimal representation of the number.
+        """
+        if self == 0:
+            return "0x00"
+        hex_str = hex(self)[2:]
+        if len(hex_str) % 2 == 1:
+            return "0x0" + hex_str
+        return "0x" + hex_str
+
+
+class HashInt(FixedSizeHexNumber[32]):  # type: ignore
+    """
+    Class that helps represent hashes in tests.
+    """
+
+    pass
+
+
 T = TypeVar("T", bound="FixedSizeBytes")