Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Service Recovery and its test case. #164

Closed
wants to merge 14 commits into from
Closed

Data Service Recovery and its test case. #164

wants to merge 14 commits into from

Conversation

yamingk
Copy link
Contributor

@yamingk yamingk commented Sep 13, 2023

Add Data Service Recovery and its test case.

The recovery flow for data service is(most of the logic are already there):

  1. load_vdev -> open_vdev
  2. vdev->add_chunk(...) -> create_blk_allocator for each chunk, each blk allocator register meta_blk_found callback for every possible meta blks possibly written before recovery boot.
  3. Start meta blk service
  4. Each BlkAllocator instance receives meta blks from MetaSvc (via pre-registered callback), and load them into memory.
  5. Recovery done.
  6. Start serving I/O requests;

A new test case added with alloc/write/free/cp_flush, then recover, followed by another round of alloc/write/cp.

Test case added and passed.

@codecov-commenter
Copy link

Codecov Report

Patch coverage: 45.00% and project coverage change: -4.71% ⚠️

Comparison is base (91b264a) 54.10% compared to head (8eac11e) 49.39%.
Report is 2 commits behind head on master.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #164      +/-   ##
==========================================
- Coverage   54.10%   49.39%   -4.71%     
==========================================
  Files          96       86      -10     
  Lines        8922     7108    -1814     
  Branches     1112      915     -197     
==========================================
- Hits         4827     3511    -1316     
+ Misses       3685     3260     -425     
+ Partials      410      337      -73     
Files Changed Coverage Δ
src/lib/blkdata_svc/blkdata_service.cpp 74.60% <ø> (ø)
src/lib/device/chunk.h 89.47% <0.00%> (-4.98%) ⬇️
src/lib/device/device_manager.cpp 76.30% <ø> (ø)
src/lib/device/journal_vdev.cpp 17.22% <ø> (ø)
src/lib/device/physical_dev.cpp 76.79% <ø> (ø)
src/lib/device/round_robin_chunk_selector.h 100.00% <ø> (ø)
src/lib/device/vchunk.cpp 0.00% <0.00%> (ø)
src/lib/device/virtual_dev.hpp 88.23% <ø> (ø)
src/lib/index/wb_cache.cpp 0.00% <ø> (ø)
src/lib/logstore/log_store_service.cpp 62.93% <ø> (ø)
... and 6 more

... and 23 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yamingk yamingk requested a review from hkadayam September 13, 2023 21:43
@yamingk yamingk changed the title Add Data Service Recovery and its test case. Data Service Recovery and its test case. Sep 13, 2023
JacksonYao287 and others added 9 commits September 16, 2023 19:55
* make VChunk CopyAssignable and CopyConstructible

* internalChunk -> m_internal_chunk
…or (#166)

* All services exposes default std::vector< BlkId > for allocating and writing blocks. This open
ended structure has difficulty for upper layers to serialize. Hence introducing MultiBlkId
which has tigther structure so upper layers can serialize and use in-place of BlkId. Existing
BlkId can still be used and some services also provide backward compatibile std::vector< BlkId >
@yamingk yamingk closed this Sep 18, 2023
@yamingk yamingk deleted the yk_recovery branch September 21, 2023 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants