Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new(tests): EOF - EIP-3540: out of order container section #741

Merged
merged 1 commit into from
Aug 26, 2024

Conversation

chfast
Copy link
Member

@chfast chfast commented Aug 11, 2024

🗒️ Description

Test containers section being out of order in the header and/or body.
This extends and follows the convention of the test_section_order()
for the optional container section.

This also converts EOFTests/efValidation/EOF1_section_order_.json.

✅ Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
  • Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
  • Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

@chfast chfast marked this pull request as draft August 11, 2024 19:01
@chfast chfast force-pushed the eof/migrate_section_order branch from 2210511 to 272db5e Compare August 26, 2024 10:28
@chfast chfast changed the title refactor(tests): use Container.expected_bytecode new(tests): EOF - EIP-3540: out of order container section Aug 26, 2024
@chfast chfast marked this pull request as ready for review August 26, 2024 10:30
@chfast chfast requested review from winsvega and marioevz August 26, 2024 10:30
@chfast chfast added scope:tests Scope: Test cases type:refactor Type: Refactor type:test Type: Test labels Aug 26, 2024
@chfast chfast force-pushed the eof/migrate_section_order branch 2 times, most recently from 551663b to e349ae9 Compare August 26, 2024 11:51
@chfast
Copy link
Member Author

chfast commented Aug 26, 2024

The coverage difference is the following:
image

This is because we "fixed" the original tests by avoiding unrelated errors: unreferenced subcontainers or subcontainers of wrong kind (LBC). The original tests had it wrong because the mentioned errors were introduced after the tests were created.
I believe the converted version of the tests is better because it exercises the happy path of subcontainer validation (GBC).

Test containers section being out of order in the header and/or body.
This extends and follows the convention of the test_section_order()
for the optional container section.

This also converts EOFTests/efValidation/EOF1_section_order_.json.
@chfast chfast force-pushed the eof/migrate_section_order branch from e349ae9 to 465f544 Compare August 26, 2024 12:41
Copy link
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks a lot for the coverage explanation!

@marioevz
Copy link
Member

This is because we "fixed" the original tests by avoiding unrelated errors: unreferenced subcontainers or subcontainers of wrong kind (LBC). The original tests had it wrong because the mentioned errors were introduced after the tests were created. I believe the converted version of the tests is better because it exercises the happy path of subcontainer validation (GBC).

Yes, I guess this will only be an issue for converted tests for which the spec has changed. Thanks for all the details on this, very helpful indeed.

@marioevz marioevz merged commit 40d8dd2 into ethereum:main Aug 26, 2024
5 of 6 checks passed
@marioevz marioevz deleted the eof/migrate_section_order branch August 26, 2024 16:31
@winsvega
Copy link
Collaborator

You see the coverage script is helpful even if it gets red.

I also found a few issues in my tests because of the cocerage report.

@chfast
Copy link
Member Author

chfast commented Aug 26, 2024

You see the coverage script is helpful even if it gets red.

I also found a few issues in my tests because of the cocerage report.

Yes, it is very helpful. I had to go back and add some missing test context multiple times. However, sometimes it may be considered unnecessary (https://github.com/ethereum/execution-spec-tests/pull/741/files#diff-ca4727af81398c427b5141ac76705400e44ba6d1afc25500f82032e1f8904756R227).

chfast added a commit to ipsilon/tests that referenced this pull request Aug 27, 2024
- EOFTests/efValidation/EOF1_embedded_container_.json ethereum/execution-spec-tests#763
- EOFTests/efValidation/EOF1_section_order_.json ethereum/execution-spec-tests#741
winsvega pushed a commit to ethereum/tests that referenced this pull request Aug 31, 2024
- EOFTests/efValidation/EOF1_embedded_container_.json ethereum/execution-spec-tests#763
- EOFTests/efValidation/EOF1_section_order_.json ethereum/execution-spec-tests#741
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:tests Scope: Test cases type:refactor Type: Refactor type:test Type: Test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants