Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flatbuffer for baseline resync #170

Merged
merged 13 commits into from
Aug 29, 2024
2 changes: 1 addition & 1 deletion conanfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

class HomeObjectConan(ConanFile):
name = "homeobject"
version = "2.0.11"
version = "2.0.12"

homepage = "https://github.com/eBay/HomeObject"
description = "Blob Store built on HomeReplication"
Expand Down
2 changes: 2 additions & 0 deletions src/lib/homestore_backend/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ settings_gen_cpp(
${CMAKE_CURRENT_BINARY_DIR}/generated/
"${PROJECT_NAME}_homestore"
hs_backend_config.fbs
resync_pg_shard.fbs
resync_blob_data.fbs
)

add_subdirectory(tests)
Expand Down
19 changes: 19 additions & 0 deletions src/lib/homestore_backend/resync_blob_data.fbs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
native_include "sisl/utility/non_null_ptr.hpp";

namespace homeobject;

table BlobData {
shard_id : uint64;
yamingk marked this conversation as resolved.
Show resolved Hide resolved
blob_id : uint64;
user_key_size : uint32;
data_size : uint32;
user_key : [ubyte];
data : [ubyte];
yamingk marked this conversation as resolved.
Show resolved Hide resolved
}

table ResyncBlobDataBatch {
data_array : [BlobData];
yamingk marked this conversation as resolved.
Show resolved Hide resolved
end_of_batch: ubyte;
}

root_type ResyncBlobDataBatch;
32 changes: 32 additions & 0 deletions src/lib/homestore_backend/resync_pg_shard.fbs
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
native_include "sisl/utility/non_null_ptr.hpp";

namespace homeobject;

table Member {
uuid : [ubyte];
name : [ubyte];
}
yamingk marked this conversation as resolved.
Show resolved Hide resolved

table PGInfoEntry {
yamingk marked this conversation as resolved.
Show resolved Hide resolved
pg_id : uint32; // only low 16 bit is used for pg_id;
priority : int32; // pg priority;
replica_set_uuid : [ubyte]; // uuid of replica set
members : [Member]; // peers;
yamingk marked this conversation as resolved.
Show resolved Hide resolved
}

table ShardInfoEntry {
state : ubyte; // shard state;
pg_id : uint32; // pg id which this shard belongs to;
yamingk marked this conversation as resolved.
Show resolved Hide resolved
shard_id : uint64; // shard id to be created with;
shard_size : uint64; // shard size;
created_time : uint64; // shard creation time
last_modified_time : uint64; // shard last modify time
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we agree to add created_lsn on shard, do we want to sync this ?
or actually setting the created_lsn to snapshot_lsn is also right, just sort of inconsistency we need to aware when we doing the scrubber.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there should be no harm adding that, let me add it.

created_lsn : int64; // lsn on shard creation;
}

table ResyncPGShardInfo {
pg : PGInfoEntry;
shards : [ShardInfoEntry];
}

root_type ResyncPGShardInfo;
Loading