[DAS] Always make expiry index #2470
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cla-bot
bot
added
the
s
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes the das filestore always create the by-expiry-timestamp directory so that expiry can be enabled at a later time if the operator desires, without needing any additional migration.
Original PR: #2385
Under the top level directory of
<datadir>/by-expiry-timestampa new directory will be created every ~2.7 hours, which, for a non-expiring DAS, gives about 3244 directories per year, giving about 20 years til we hit the 65K we are trying to avoid.Example path:
<datadir>/by-expiry-timestamp/171702/5712/1b7a84fcdb5f467ec4889e99dba58b2e6d56e0154538ee2a3083aa1582ab833eTesting done
Re-tested migrating the old style format with the configuration used in #2385 but with
data-availability.local-file-storage.enable-expiryset to false, and spot checked that theby-expiry-timestampdirectory was populated correctly. Stopped the daserver and checked that settingdata-availability.local-file-storage.enable-expiryto true then pruned the data as expected.