Remove logstore family #281

sanebay · 2024-01-19T18:46:21Z

Earlier logstore services have two fixed logstore families, 1 logdev under each family and family managing multiple logstores . This PR removes the logstore families indirection. Now we remove the logstore family class and let logstore service handles multiple logdev's and logdevs handle multiple logstores under it. Clients can now create multiple log devices each having its own state. This PR depends on the dynamic journal vdev PR. Earlier there were fixed number of metablks. Now we can have more metablks, one per logdev.

Earlier
       LogStoreService 
             LogStoreFamily(CTRL and DATA)
                          LogStores  -> 1 LogDev
Now
        LogStoreService 
             LogDev(1... N)
                      LogStores(1..N). multiple logstores under 1 LogDev.

Removed the LOG_REPLICATED and LOG_LOCAL and make it single LOG service.
Move functionality of log store family of handling log stores to logdev class.
We have a single journal vdev. All logdevices uses journal descriptors to talk to a single journal vdev. Each descriptor has its own list of chunks and state of the offset from a single journal vdev.
Added api's to create/open logdev in logservice.

src/include/homestore/logstore/log_store_internal.hpp

src/include/homestore/logstore_service.hpp

src/lib/logstore/log_dev.cpp

src/include/homestore/logstore/log_store_internal.hpp

src/include/homestore/logstore_service.hpp

src/lib/logstore/log_dev.cpp

src/tests/test_common/hs_repl_test_common.hpp

hkadayam

Have some minor comments, but don't want to hold this PR merge. I am fine in merge this and we can adjust minor things in next checkin.

Remove the fixed number of logstore families and instead dynamically create logdev and logstores under them. Logservice managers logdev's and logdev manages logstores. Create a single journal vdev and multiple logdev use the journal vdev with journal descriptor maintaing the state. Remove LOG_REPLICATED and LOG_LOCAL and use LOG service.

codecov-commenter · 2024-02-05T21:39:31Z

Codecov Report

Attention: 89 lines in your changes are missing coverage. Please review.

Comparison is base (4bbea89) 60.95% compared to head (371c2d9) 61.04%.
Report is 3 commits behind head on master.

Files	Patch %	Lines
src/lib/logstore/log_dev.cpp	76.04%	30 Missing and 10 partials ⚠️
src/lib/logstore/log_store_service.cpp	77.47%	23 Missing and 2 partials ⚠️
.../lib/replication/log_store/home_raft_log_store.cpp	0.00%	13 Missing ⚠️
src/lib/replication/repl_dev/raft_repl_dev.cpp	0.00%	5 Missing ⚠️
src/include/homestore/homestore.hpp	0.00%	1 Missing and 1 partial ⚠️
src/lib/homestore.cpp	75.00%	0 Missing and 2 partials ⚠️
src/lib/logstore/log_store.cpp	94.11%	1 Missing ⚠️
...rc/lib/replication/log_store/home_raft_log_store.h	0.00%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #281      +/-   ##
==========================================
+ Coverage   60.95%   61.04%   +0.08%     
==========================================
  Files         109      107       -2     
  Lines        9182     9197      +15     
  Branches     1185     1173      -12     
==========================================
+ Hits         5597     5614      +17     
- Misses       3052     3055       +3     
+ Partials      533      528       -5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sanebay changed the title ~~Remove log family~~ Remove logstore family Jan 19, 2024

sanebay force-pushed the remove_log_family branch from caeb941 to 4b45a01 Compare January 23, 2024 19:25

sanebay requested review from hkadayam and yamingk January 23, 2024 19:25

hkadayam reviewed Jan 25, 2024

View reviewed changes

src/include/homestore/logstore/log_store_internal.hpp Outdated Show resolved Hide resolved

sanebay force-pushed the remove_log_family branch from 4b45a01 to a491acc Compare January 25, 2024 19:45