Allow separate cache in / out directories #97
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hey!
We use the
oci-build-task
quite heavily, but have noticed that over time thecache
folder grows as our Dockerfiles are changed, and older layers are not removed when they are no longer relevant to the built image. This can cause the moving of the cache between workers to take more time than is saved by using the cache, which somewhat defeats the purpose.We could use the
caches
directive to only keep the cache on the worker the task is run on, but this leads to rebuilding the image entirely whenever the task lands on a different worker. Without sharing the cache between workers, image layers would be re-built and duplicated in our container image repository too (even when the underlying files have not changed).This PR adds
cache-in
andcache-out
folders to the task. If acache
folder is not included in the task configuration, then these folders will be checked to see if they exist and, if so, used instead of thecache
folder.As the name implies,
cache-in
is where the cache will be imported from, andcache-out
is where it will be exported to. Exporting the cache will (should?) only include the relevant layers, keeping the cache from including unnecessary layers and growing over time.