Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Contract Pruning #753

Merged
merged 73 commits into from
Dec 6, 2023
Merged

Contract Pruning #753

merged 73 commits into from
Dec 6, 2023

Conversation

peterjan
Copy link
Member

@peterjan peterjan commented Nov 21, 2023

This PR adds contract pruning to the contractor. I'm following it up with a metrics PR. I've been testing it quite extensively on a separate (integrity) node and haven't ran into any issues. I even discovered a nasty race in the upload manager wrt uploading sectors. I chose to extend the upload manager with persistence, to avoid having the caller worry about "finishing" the upload after persisting whatever needs persisting in the bus. It passed 1200 iterations of TestSectorPruning so should be good.

@peterjan peterjan self-assigned this Nov 21, 2023
api/autopilot.go Outdated Show resolved Hide resolved
@peterjan peterjan marked this pull request as ready for review November 23, 2023 09:43
api/autopilot.go Outdated Show resolved Hide resolved
autopilot/contractor.go Outdated Show resolved Hide resolved
@peterjan
Copy link
Member Author

peterjan commented Dec 4, 2023

testing on areq - will merge tomorrow morning if things did not explode

@peterjan
Copy link
Member Author

peterjan commented Dec 5, 2023

@alexfreska this PR adds a new field to the ContractsConfig called "Prune".

When set to true, the autopilot will try and prune deleted sector roots from contracts, one contract at a time, for a max duration of 10 minutes per contract. For old hosts this process takes quite a while, for new hosts it's fast and will make it so deleted data is also effectively deleted from the contract, making sure renters don't keep paying for storage they are not using.

Eventually this can/should be tied in with prune metrics, and perhaps also expose /api/bus/contracts/prunable data in the UI (e.g. together with the object stats, alongside reclaimable storage maybe). For now though, could we add this to the autopilot configuration and default it to false?

@peterjan peterjan merged commit 4680be0 into master Dec 6, 2023
6 checks passed
@peterjan peterjan deleted the pj/contract-pruning branch December 6, 2023 15:30
@peterjan peterjan restored the pj/contract-pruning branch August 27, 2024 08:24
@ChrisSchinnerl ChrisSchinnerl deleted the pj/contract-pruning branch December 19, 2024 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants