Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shove fast #692

Merged
merged 3 commits into from
Feb 11, 2024
Merged

shove fast #692

merged 3 commits into from
Feb 11, 2024

Conversation

nalinbhardwaj
Copy link
Member

No description provided.

Copy link
Member

@dcposch dcposch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@dcposch
Copy link
Member

dcposch commented Feb 10, 2024

Follow-up request / api req counts in shovel logs

current

p=1 v=436c chain=84532 src=base-sepolia dst=transfers n=5918582 h= nrows=12 delta=1 elapsed=17.801278ms msg=converge

would love to get a something like this

p=1 v=436c chain=84532 src=base-sepolia dst=transfers n=5918582 h= nrows=12 delta=1 napi=1 elapsed=17ms msg=converge

vs, say:

p=1 v=436c chain=84532 src=base-sepolia dst=transfers n=5918582 h= nrows=12 delta=1 napi=13 elapsed=500ms msg=converge

show num node api calls. this'll show whether we're doing an n+1 query (loading block bodies vs just headers), good sanity check

@ryandotsmith
Copy link
Contributor

Follow-up request / api req counts in shovel logs

Great suggestion. Will do.

@dcposch
Copy link
Member

dcposch commented Feb 11, 2024

One more idea

BEHIND 550 BLOCKS ETA=7min p=1 v=436c chain=84532 src=base-sepolia dst=transfers tip=5918582 nblocks=100 nrows=272 napi=1 elapsed=500ms

vs

READY p=1 v=436c chain=84532 src=base-sepolia dst=transfers tip=5918582 nblocks=1 nrows=18 napi=1 elapsed=50ms

READY p=1 v=436c chain=84532 src=base-sepolia dst=transfers tip=5918582 nblocks=8 NREORG=7 nrows=18 napi=1 elapsed=50ms

  • status upfront
  • bonus points for reorg display
  • & just to confirm: this line prints after nrows were successfully written to pg, right?

@dcposch dcposch merged commit a19578d into master Feb 11, 2024
5 checks passed
@ryandotsmith
Copy link
Contributor

  • & just to confirm: this line prints after nrows were successfully written to pg, right?

Correct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants