Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and Improve fix pg size & shard identical layout #235

Merged

Conversation

Hooper9973
Copy link
Contributor

Bug Fixes:

  • Resolved issues in create_shard and seal_shard functions to ensure correct shard management. Enhancements:
  • Added validation to prevent creation of placement groups with sizes smaller than the chunk size. Test Improvements:
  • pg test:
    • Added tests to verify that creating a placement group with a size smaller than the chunk size results in failure.
  • shard test:
    • Created shard but shard size is larger than pg space left which should be failed.
    • Verified that successfully created shards have identical virtual chunk layouts.

@Hooper9973 Hooper9973 force-pushed the improve_pg_size_and_identical_layout_UT branch from 0bfe44f to 670ad77 Compare December 2, 2024 09:35
Bug Fixes:
- Resolved issues in create_shard and seal_shard functions to ensure correct shard management.
Enhancements:
- Added validation to prevent creation of placement groups with sizes smaller than the chunk size.
Test Improvements:
- pg test:
	- Added tests to verify that creating a placement group with a size smaller than the chunk size results in failure.
- shard test:
	- Created shard but shard size is larger than pg space left which should be failed.
	- Verified that successfully created shards have identical virtual chunk layouts.
@Hooper9973 Hooper9973 force-pushed the improve_pg_size_and_identical_layout_UT branch from 670ad77 to 6e5e52e Compare December 3, 2024 00:41
@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 64.70588% with 6 lines in your changes missing coverage. Please review.

Project coverage is 65.30%. Comparing base (1746bcc) to head (6e5e52e).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
...ib/homestore_backend/replication_state_machine.cpp 0.00% 2 Missing and 1 partial ⚠️
src/lib/homestore_backend/heap_chunk_selector.cpp 50.00% 1 Missing and 1 partial ⚠️
src/lib/homestore_backend/hs_shard_manager.cpp 85.71% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #235      +/-   ##
==========================================
+ Coverage   63.15%   65.30%   +2.14%     
==========================================
  Files          32       32              
  Lines        1900     1977      +77     
  Branches      204      213       +9     
==========================================
+ Hits         1200     1291      +91     
+ Misses        600      578      -22     
- Partials      100      108       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -56,6 +57,7 @@ class HSReplTestHelper {
sync_point_num_ = sync_point;
homeobject_replica_count_ = 0;
uint64_id_ = INVALID_UINT64_ID;
v_chunk_id_ = INVALID_CHUNK_NUM;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: I prefer to cast homestore::chunk_num_t(uinit_16) to uint_64 here, and then cast it back.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll merge first, and if necessary, I will refine in next pr.
BTW, what are the pros in this way? 😃

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hs_repl_test_helper.hpp is the framework, it does not know what is a shard_id_t , pg_id_t , blob_id_t and chunk_id_t. these types are only know application layer.

if we add v_chunk_id here, should we also add shard_id_t, pg_id_t and others?

so I think we can use this 64bit as a common shared data across different processed, and application layer should translate this to what it wants

Copy link
Collaborator

@JacksonYao287 JacksonYao287 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, only a minor comment

@Hooper9973 Hooper9973 merged commit 89f32a6 into eBay:main Dec 3, 2024
25 checks passed
@Hooper9973 Hooper9973 deleted the improve_pg_size_and_identical_layout_UT branch December 3, 2024 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants