Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pull in geth changes for state recreation #2005

Merged
merged 49 commits into from
Mar 13, 2024
Merged

pull in geth changes for state recreation #2005

merged 49 commits into from
Mar 13, 2024

Conversation

magicxyyz
Copy link
Contributor

@magicxyyz magicxyyz commented Dec 7, 2023

@cla-bot cla-bot bot added the s Automatically added by the CLA bot if the creator of a PR is registered as having signed the CLA. label Dec 7, 2023
@magicxyyz magicxyyz changed the title pull in geth changes for state recreation and adjust tests pull in geth changes for state recreation Dec 7, 2023
@magicxyyz magicxyyz marked this pull request as ready for review December 8, 2023 16:19
@joshuacolvin0 joshuacolvin0 requested a review from tsahee December 8, 2023 19:10
@yucem44 yucem44 mentioned this pull request Dec 16, 2023
amsanghi
amsanghi previously approved these changes Dec 16, 2023
amsanghi
amsanghi previously approved these changes Dec 19, 2023
@magicxyyz magicxyyz marked this pull request as ready for review December 19, 2023 17:02
Copy link
Contributor

@tsahee tsahee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved with comments on tests.
Let's do a round on these in chat / separate PR.


blockCountRequiredToFlushDirties := builder.execConfig.Caching.BlockCount
makeSomeTransfers(t, ctx, builder, blockCountRequiredToFlushDirties)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add here a check that StateAndHeaderByNumber will now fail?
Or maybe force state to drop after the end of current test - and check that StateAndHeaderByNumber fails then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in #2442

}
}

func TestGettingStateForRPCHybridArchiveNode(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the logic here different from above other than config? can we merge the functions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in #2442

var wgCallers sync.WaitGroup
for j := 0; j < threads && ctx.Err() == nil; j++ {
wgCallers.Add(1)
go func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this gofunc is quite weird and deserves at least some explanation in comments

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in #2442

}()
api := builder.L2.ExecNode.Backend.APIBackend()
db := builder.L2.ExecNode.Backend.ChainDb()
i := 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a sync variable between multiple threads.. the name "i" doesn't really fit it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in #2442

prefix = append([]byte("b"), prefix...)
it := db.NewIterator(prefix, nil)
defer it.Release()
if it.Next() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: to speed up polling and hit more edge-cases, each thread could cache blockhashes that it already got a non-error response for in previous loops and skip them the next time iterator gets them

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alternatively, we could poll old blockhashes and make sure they don't return the "ahead of current block" error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test is mostly intended to be a regression test for a specific issue, I added a comment describing the details in the follow up PR: #2442

there should be only one key with the prefix (unless there was a reorg) so there is no need of caching. If the entry is found, the recentBlock (previously i) is advanced to the next block, and the iterator is created for the new prefix.

Let me know if you think that extending this test makes sense to cover more cases then the targeted issue

@tsahee tsahee merged commit d1b4201 into master Mar 13, 2024
8 checks passed
@tsahee tsahee deleted the state-release branch March 13, 2024 01:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
geth Involves changes to Go-Ethereum s Automatically added by the CLA bot if the creator of a PR is registered as having signed the CLA.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants