Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: rename to blob hub #3

Merged
merged 1 commit into from
Apr 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: Report a bug
about: Something with blob-syncer is not working as expected
about: Something with blob-hub is not working as expected
title: ''
labels: 'type:bug'
assignees: ''
Expand Down
18 changes: 9 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,34 +1,34 @@
VERSION=$(shell git describe --tags)
GIT_COMMIT=$(shell git rev-parse HEAD)
GIT_COMMIT_DATE=$(shell git log -n1 --pretty='format:%cd' --date=format:'%Y%m%d')
REPO=github.com/bnb-chain/blob-syncer
IMAGE_NAME=ghcr.io/bnb-chain/blob-syncer
REPO=github.com/bnb-chain/blob-hub
IMAGE_NAME=ghcr.io/bnb-chain/blob-hub

ldflags = -X $(REPO)/version.AppVersion=$(VERSION) \
-X $(REPO)/version.GitCommit=$(GIT_COMMIT) \
-X $(REPO)/version.GitCommitDate=$(GIT_COMMIT_DATE)

build_syncer:
ifeq ($(OS),Windows_NT)
go build -o build/blob-syncer.exe -ldflags="$(ldflags)" cmd/blob-syncer/main.go
go build -o build/syncer.exe -ldflags="$(ldflags)" cmd/blob-hub-syncer/main.go
else
go build -o build/blob-syncer -ldflags="$(ldflags)" cmd/blob-syncer/main.go
go build -o build/syncer -ldflags="$(ldflags)" cmd/blob-hub-syncer/main.go
endif

build_server:
ifeq ($(OS),Windows_NT)
go build -o build/blob-syncer-server.exe -ldflags="$(ldflags)" cmd/blob-syncer-server/main.go
go build -o build/server.exe -ldflags="$(ldflags)" cmd/blob-hub-server/main.go
else
go build -o build/blob-syncer-server -ldflags="$(ldflags)" cmd/blob-syncer-server/main.go
go build -o build/server -ldflags="$(ldflags)" cmd/blob-hub-server/main.go
endif

build:
make build_syncer
make build_server

install:
go install cmd/blob-syncer/main.go
go install cmd/blob-syncer-server/main.go
go install cmd/blob-hub-syncer/main.go
go install cmd/blob-hub-server/main.go

build_docker:
docker build . -t ${IMAGE_NAME}
Expand Down Expand Up @@ -59,6 +59,6 @@ format:
.PHONY: lint lint-fix format

swagger-gen:
swagger generate server -f ./swagger.yaml -A blob-syncer --default-scheme=http
swagger generate server -f ./swagger.yaml -A blob-hub --default-scheme=http


18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# Blob Syncer
# Blob Hub

## Overview

The Blob-Syncer service acts as an intermediary between Greenfield and Ethereum/BSC, continuously retrieving blobs from Ethereum/BSC and persisting in Greenfield..
The blob-hub service acts as an intermediary between Greenfield and Ethereum/BSC, continuously retrieving blobs from Ethereum/BSC and persisting in Greenfield..
These blobs, collected from a specific Beacon slot range on Ethereum or specific block height range on BSC, are consolidated into a bundle and transmitted to the [Bundle Service](https://docs.bnbchain.org/greenfield-docs/docs/api/bundle-service/)
for further processing. Subsequently, the bundle is stored in Greenfield for long-term archival purposes. Additionally,
the Blob-Syncer service offers APIs that enable users to access and retrieve historical blobs as needed.
the blob-hub service offers APIs that enable users to access and retrieve historical blobs as needed.

## Disclaimer
- Data Accessibility. Due to the limited size of the Greenfield traffic package purchased, this service does not guarantee constant data access but ensures eventual access.
- Data latency. As the blob syncer need to bundle blobs across multi blocks, the latency is around 10-20 minutes before the blob is accessible on Greenfield.

## Components
- **blob-syncer**: This component is specifically engineered to synchronize blobs and securely store them in Greenfield. It includes a post-verification process to ensure the integrity of blob storage.
- **blob-syncer-server**: This component functions as the API server, catering to users' requests for querying blobs.
- **syncer**: This component is specifically engineered to synchronize blobs and securely store them in Greenfield. It includes a post-verification process to ensure the integrity of blob storage.
- **server**: This component functions as the API server, catering to users' requests for querying blobs.

## Support Network
| Network | Endpoint | Bucket |
Expand Down Expand Up @@ -202,7 +202,7 @@ make build
make build_syncer
```

### Build blob syncer api server
### Build Blob Hub api server

```shell
make build_server
Expand All @@ -213,7 +213,7 @@ make build_server
### Run the Blob Syncer instance

```shell
./build/blob-syncer --config-path config/local/config-syncer.json
./build/syncer --config-path config/local/config-syncer.json
```

```json
Expand All @@ -236,7 +236,7 @@ make build_server
"dialect": "mysql",
"username": "root",
"password": "pass",
"url": "/blob-syncer?charset=utf8&parseTime=True&loc=Local",
"url": "/blob-hub?charset=utf8&parseTime=True&loc=Local",
"max_idle_conns": 10,
"max_open_conns": 100
},
Expand All @@ -260,5 +260,5 @@ make build_server
### Run the api server

```shell
./build/blob-syncer-server --config-path config/local/config-server.json --port 8080
./build/server --config-path config/local/config-server.json --port 8080
```
10 changes: 5 additions & 5 deletions cmd/blob-syncer-server/main.go → cmd/blob-hub-server/main.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions cmd/blob-syncer/main.go → cmd/blob-hub-syncer/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ import (
"github.com/spf13/pflag"
"github.com/spf13/viper"

"github.com/bnb-chain/blob-syncer/config"
syncerdb "github.com/bnb-chain/blob-syncer/db"
"github.com/bnb-chain/blob-syncer/logging"
"github.com/bnb-chain/blob-syncer/metrics"
"github.com/bnb-chain/blob-syncer/syncer"
"github.com/bnb-chain/blob-hub/config"
syncerdb "github.com/bnb-chain/blob-hub/db"
"github.com/bnb-chain/blob-hub/logging"
"github.com/bnb-chain/blob-hub/metrics"
"github.com/bnb-chain/blob-hub/syncer"
)

func initFlags() {
Expand Down
4 changes: 2 additions & 2 deletions config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ import (
"gorm.io/gorm"
"gorm.io/gorm/logger"

"github.com/bnb-chain/blob-syncer/cache"
syncerdb "github.com/bnb-chain/blob-syncer/db"
"github.com/bnb-chain/blob-hub/cache"
syncerdb "github.com/bnb-chain/blob-hub/db"
)

type SyncerConfig struct {
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
module github.com/bnb-chain/blob-syncer
module github.com/bnb-chain/blob-hub

go 1.20

Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bits-and-blooms/bitset v1.11.0 h1:RMyy2mBBShArUAhfVRZJ2xyBO58KCBCtZFShw3umo6k=
github.com/bits-and-blooms/bitset v1.11.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8=
github.com/bnb-chain/blob-syncer v0.0.1 h1:GImArcAzx23ntV2l8dOrPJX11YiAoMWXsRh3JLfHQKc=
github.com/bnb-chain/blob-syncer v0.0.1/go.mod h1:7JAEkO3ID/Kj56eyRV4SpiIf2vi1/czOYrlv3Z7XMBQ=
github.com/bnb-chain/greenfield-bundle-sdk v1.1.0 h1:0BWQsV+c32wHxEEpJY9igBSBg5N1Fm3KoSLC+Yef2n0=
github.com/bnb-chain/greenfield-bundle-sdk v1.1.0/go.mod h1:NCjQp0sniAbBR5yR5pYiXpYwYd1okSIBLj+31sTpmXA=
github.com/btcsuite/btcd v0.23.3 h1:4KH/JKy9WiCd+iUS9Mu0Zp7Dnj17TGdKrg9xc/FGj24=
Expand Down
7 changes: 4 additions & 3 deletions logging/log.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@ package logging
import (
"os"

"github.com/op/go-logging"
"gopkg.in/natefinch/lumberjack.v2"

"github.com/bnb-chain/blob-syncer/config"
"github.com/op/go-logging"

"github.com/bnb-chain/blob-hub/config"
)

var (
// Logger instance for quick declarative logging levels
Logger = logging.MustGetLogger("blob-syncer")
Logger = logging.MustGetLogger("blob-hub")
// log levels that are available
levels = map[string]logging.Level{
"CRITICAL": logging.CRITICAL,
Expand Down
2 changes: 1 addition & 1 deletion metrics/metric.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"

"github.com/bnb-chain/blob-syncer/logging"
"github.com/bnb-chain/blob-hub/logging"
)

var (
Expand Down
23 changes: 12 additions & 11 deletions restapi/configure_blob_syncer.go → restapi/configure_blob_hub.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,35 +7,35 @@ import (
"net/http"
"os"

"github.com/bnb-chain/blob-hub/cache"
"github.com/bnb-chain/blob-hub/config"
syncerdb "github.com/bnb-chain/blob-hub/db"
"github.com/bnb-chain/blob-hub/external"
"github.com/bnb-chain/blob-hub/restapi/handlers"
"github.com/bnb-chain/blob-hub/service"
"github.com/go-openapi/errors"
"github.com/go-openapi/runtime"
"github.com/go-openapi/swag"

"github.com/bnb-chain/blob-syncer/cache"
"github.com/bnb-chain/blob-syncer/config"
syncerdb "github.com/bnb-chain/blob-syncer/db"
"github.com/bnb-chain/blob-syncer/external"
"github.com/bnb-chain/blob-syncer/restapi/handlers"
"github.com/bnb-chain/blob-syncer/restapi/operations"
"github.com/bnb-chain/blob-syncer/restapi/operations/blob"
"github.com/bnb-chain/blob-syncer/service"
"github.com/bnb-chain/blob-hub/restapi/operations"
"github.com/bnb-chain/blob-hub/restapi/operations/blob"
)

//go:generate swagger generate server --target ../../blob-syncer --name BlobSyncer --spec ../swagger.yaml --principal interface{}
//go:generate swagger generate server --target ../../blob-syncer --name BlobHub --spec ../swagger.yaml --principal interface{}

var cliOpts = struct {
ConfigFilePath string `short:"c" long:"config-path" description:"Config path" default:""`
}{}

func configureFlags(api *operations.BlobSyncerAPI) {
func configureFlags(api *operations.BlobHubAPI) {
param := swag.CommandLineOptionsGroup{
ShortDescription: "config",
Options: &cliOpts,
}
api.CommandLineOptionsGroups = []swag.CommandLineOptionsGroup{param}
}

func configureAPI(api *operations.BlobSyncerAPI) http.Handler {
func configureAPI(api *operations.BlobHubAPI) http.Handler {
// configure the api here
api.ServeError = errors.ServeError

Expand Down Expand Up @@ -102,6 +102,7 @@ func configureServer(s *http.Server, scheme, addr string) {
panic("currently only local cache is support.")
}
service.BlobSvc = service.NewBlobService(blobDB, bundleClient, cacheSvc, cfg)

}

// The middleware configuration is for the handler executors. These do not apply to the swagger.json document.
Expand Down
6 changes: 3 additions & 3 deletions restapi/doc.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions restapi/embedded_spec.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions restapi/handlers/blob.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ import (
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/go-openapi/runtime/middleware"

"github.com/bnb-chain/blob-syncer/models"
"github.com/bnb-chain/blob-syncer/restapi/operations/blob"
"github.com/bnb-chain/blob-syncer/service"
"github.com/bnb-chain/blob-syncer/util"
"github.com/bnb-chain/blob-hub/models"
"github.com/bnb-chain/blob-hub/restapi/operations/blob"
"github.com/bnb-chain/blob-hub/service"
"github.com/bnb-chain/blob-hub/util"
)

const rootLength = 32
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading