Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vine: Add symlink mount option. #3527

Merged
merged 12 commits into from
Oct 3, 2023

Conversation

dthain
Copy link
Member

@dthain dthain commented Oct 2, 2023

Proposed changes

Attaching large directory trees to the sandbox is expensive by default b/c the worker does a recursive hard-link of the entire directories. (This is to avoid the tasking being able to "tell" that the object is linked in.)

This PR adds a mount option allowing the object to be attached by a single symlink instead, if the user asks for it.

Post-change actions

Put an 'x' in the boxes that describe post-change actions that you have done.
The more 'x' ticked, the faster your changes are accepted by maintainers.

  • make test Run local tests prior to pushing.
  • make format Format source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)
  • make lint Run lint on source code prior to pushing.
  • Manual Update Did you update the manual to reflect your changes, if appropriate? This action should be done after your changes are approved but not merged.
  • Type Labels Select github labels for the type of this change: bug, enhancement, etc.
  • Product Labels Select github labels for the product affected: TaskVine, Makeflow, etc.
  • PR RTM Mark your PR as ready to merge.

Additional comments

Considering whether VINE_MOUNT_SYMLINK should be used by default in add_environment, which would cover most of the poncho use cases.

@dthain dthain merged commit 2f558c8 into cooperative-computing-lab:master Oct 3, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants