This is a top-level issue for tracking the manual porting of ethereum/tests test cases to execution-spec-tests:

Contributions to manually port individual test cases are very welcome!

If you're already familiar with ethereum/tests and execution-spec-tests, jump to the issue tracker below!

Background

ethereum/tests was the repository used to define EVM test cases from Frontier up to and including The Merge. The test cases are defined as YAML (and sometimes JSON) files in the ./src/ sub-directory and the JSON "test fixtures" (fully-populated tests that can be executed against clients) are generated using ethereum/retesteth. These JSON artifacts are regenerated when required and added to the repository, for example, in the ./GeneralStateTests/ sub-directory.

From Shanghai on, new test cases (in particular for new features included in hard-forks) have been defined in Python in this repository (execution-spec-tests). The existing test cases are still important to execute against clients, however, and porting ethereum/tests to execution-spec-tests ensures that the test cases can be continually maintained and filled (generated) for newer forks. It also has the advantage that client teams will only need to obtain test fixture releases from one source.

Automated Port to Python

Although there will be an effort to automate ("transpile") the remaining test cases from YAML (respectively JSON) to Python; contributions to manually port individual test cases can be very beneficial and are welcome!

The benefits of manual porting are:

1. Reduce the number of test cases by combing multiple YAML (resp. JSON) test cases in one Python test function (via parametrization).
2. Potentially improve coverage by parametrizing the Python version.
3. Code/docstrings are likely higher quality than an automated conversion.
4. The test can be better organized within the ./tests folder in execution-spec-tests by the fork and the EIP in which the functionality was introduced.

Process

1. Pick a single or a set of YAML test cases in ethereum/tests to port and create an issue in this repository - optional, but definitely recommended if porting tests from the tracker below to avoid duplicate work.
2. Create a PR with the ported tests:
   a. Add the list of ported YAML files to converted-ethereum-tests.txt.
   b. If the tests can't currently be filled, please explain the issue (feel free to also open a Discussion).

Help with ethereum/tests

The test formats are described in the ethereum/tests online docs. Note that state filler test cases in YAML can be parametrized, see the docs here.

Hints to Get Started with ethereum/execution-spec-tests

0. Start here: Getting Started → Installation.
1. Use uv run et make test to create a template test module (currently a WIP in ✨ feat(et): add et make test for interactively creating new tests #950).
2. Where to put the test? See how tests are organized in Writing Tests → Adding a New Test.
3. State vs Blockchain test? If in doubt try to write a state test, see Writing Tests → Types of Tests → Deciding on a Test Type.
4. Tests are filled via EELS by default (no further installation necessary; it's a Python dependency and got installed in Step 0), see Filling → Getting Started.
5. Parametrizing test functions is encouraged, see Writing Tests → Writing a New Test → Parametrizing Tests.
6. The Opcode mini-lang defined in this repository is the preferred way to define bytecode in tests (if feasible). For a very simple example, see test_chainid() - there are other examples in the tests.

Feel free to reach out for help or guidance in the Ethereum R&D discord or via DM; see Getting Help in the docs.

Some Examples of Ported Tests