Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Blob Syncer #1

Merged
merged 20 commits into from
Apr 19, 2024
Merged

feat: Blob Syncer #1

merged 20 commits into from
Apr 19, 2024

Conversation

alexgao001
Copy link
Collaborator

@alexgao001 alexgao001 commented Apr 3, 2024

The blob-syncer is the service sitting between Greenfield and Ethereum, constantly fetching blobs from Ethereum and persisting them into Greenfield storage for permanent archiving. The blob-syncer service also provides APIs for users to query historical blobs.

Makefile Show resolved Hide resolved
README.md Outdated
./build/blob-syncer --config-type local --config-path config/config.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please draft a decent readme before the PR is fully ready.

config/config.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
config/config.json Outdated Show resolved Hide resolved
config/config.json Outdated Show resolved Hide resolved
config/utils.go Outdated Show resolved Hide resolved
db/blob.go Outdated Show resolved Hide resolved
db/block.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
swagger.yaml Outdated
- http

paths:
/beacon/blob_sidecars/{blockNum}:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the Beacon Chain API spec, it is {block Id} here , not {block number}, these two are different meanings, can we make it 100% compatible?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

swagger.yaml Show resolved Hide resolved
syncer/blob_syncer.go Outdated Show resolved Hide resolved
syncer/blob_syncer.go Outdated Show resolved Hide resolved
forkedBlock := &db.Block{
Height: nextHeight,
}
return l.blobDao.SaveBlockAndBlob(forkedBlock, nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we save such a block? I thought we dont save any data if it is a possible forked block.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

save it only to track the process block. As the slot number will not be reused in beacon chain.

syncer/blob_syncer.go Outdated Show resolved Hide resolved
syncer/blob_syncer.go Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
cmd/blob-syncer/main.go Outdated Show resolved Hide resolved
syncer/syncer.go Outdated Show resolved Hide resolved
syncer/syncer.go Outdated Show resolved Hide resolved
syncer/syncer.go Outdated Show resolved Hide resolved
syncer/syncer.go Outdated Show resolved Hide resolved
}

if isForkedBlock {
return s.blobDao.SaveBlockAndBlob(&db.Block{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to save it if it is a forked block?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when the syncer init a bundle, thhe bundle name will be binded with (startSlot, endSlot), the endSlot could be a forked block later. saving a forked block and treat it as a non-blobs included block will make the verifying process easier. as it need to query forked block from DB.

syncer/verifier.go Outdated Show resolved Hide resolved
@alexgao001 alexgao001 force-pushed the blob-syncer branch 2 times, most recently from acd4524 to 1c7107b Compare April 17, 2024 12:44
@unclezoro unclezoro merged commit ee6f064 into master Apr 19, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants