Add long running log store tests.

[sanebay]

Changes.

1. Fix the truncation in corner cases.
2. Add long running test for logstore. Stripped down version of test_log_store. Add burst of requests to all logstores, truncate, restart, add/remove logstores logdev's, ad rollbacks etc.
3. Enable logstore test except the parallel truncation UT.

More details.
Journalvdev maintains a list of chunks to store all the log entries. All append log entries are appended to the last chunk in the list(right side/tail offset) and truncate is applied to the head of the chunk list(left part/ data start offset). Whenever we append log entries, if we dont have enough space, we create a new chunk and append to the list. So log groups(batch of log entries) dont go across chunks. So there will be holes in these chunks at the end which are marked by end_of_chunk in chunk private data. The hole lies between end_of_chunk and (chunk_start + chunk_size) . When we read and we reach this hole, there is no data so we skip and move to the next chunk. Similarly if truncate happens to be in that hole also, we release the whole chunk and move to the next chunk. Also we set the data_start_offset to the start of the next chunk.

[yamingk]

We also need to add flip point when we add new chunks to the journal_vdev, in which we do persistence of the private data and think it through what will happen on reboot and verify with a test case.

[yamingk]

Ideally we should have a long running either at log store level or journal vdev level, that we do truncate periodically for a few hours and make sure it is running fine.
Another test case is run journal or logstore test for 30 mins (with trunation happening more frequently), do a clean shutdown and do another 15 mins run and repeat clean shutdown for multiple times.

We can review these and see if there is anything comment from others and create issues (not necessaries to be included in this PR).

[sanebay]

Ideally we should have a long running either at log store level or journal vdev level, that we do truncate periodically for a few hours and make sure it is running fine. Another test case is run journal or logstore test for 30 mins (with trunation happening more frequently), do a clean shutdown and do another 15 mins run and repeat clean shutdown for multiple times.

We can review these and see if there is anything comment from others and create issues (not necessaries to be included in this PR).

test_log_store_long_run.cpp is doing that what you mentioned. Will have to run it on 85 namespace.

[sanebay]

We also need to add flip point when we add new chunks to the journal_vdev, in which we do persistence of the private data and think it through what will happen on reboot and verify with a test case.

Added this in the doc.

[yamingk]

Ideally we should have a long running either at log store level or journal vdev level, that we do truncate periodically for a few hours and make sure it is running fine. Another test case is run journal or logstore test for 30 mins (with trunation happening more frequently), do a clean shutdown and do another 15 mins run and repeat clean shutdown for multiple times.
We can review these and see if there is anything comment from others and create issues (not necessaries to be included in this PR).

test_log_store_long_run.cpp is doing that what you mentioned. Will have to run it on 85 namespace.

Right, is the truncation point ramdomized?

[yamingk]

We also need to add flip point when we add new chunks to the journal_vdev, in which we do persistence of the private data and think it through what will happen on reboot and verify with a test case.

Added this in the doc.

Better to create issue otherwise it will lose track which are already there which are todos.

[sanebay]

Ideally we should have a long running either at log store level or journal vdev level, that we do truncate periodically for a few hours and make sure it is running fine. Another test case is run journal or logstore test for 30 mins (with trunation happening more frequently), do a clean shutdown and do another 15 mins run and repeat clean shutdown for multiple times.
We can review these and see if there is anything comment from others and create issues (not necessaries to be included in this PR).

test_log_store_long_run.cpp is doing that what you mentioned. Will have to run it on 85 namespace.

Right, is the truncation point ramdomized?

Yes test_log_store_long_run.cpp:459.

[sanebay]

Created a ticket to use version instead of created time (#441)