Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Vine Shared Filesystem Mode #3944

Closed

Conversation

dthain
Copy link
Member

@dthain dthain commented Sep 27, 2024

Proposed Changes

Added new access mode enabled by vine_tune() to compare TaskVine caching behavior to direct use of shared filesystem.

  • Add VINE_SHAREDFS file type which causes worker to link from sandbo…x to shared filesystem.
  • Added shared_filesystem_link_mode to replace VINE_FILE with VINE_SHAREDFS.
  • Rename load_from_shared_fs to shared_filesystem_cache_mode, which replaces VINE_FILE with VINE_URL.

Merge Checklist

The following items must be completed before PRs can be merge.
Check these off to verify you have completed all steps.

  • make test Run local tests prior to pushing.
  • make format Format source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)
  • make lint Run lint on source code prior to pushing.
  • Manual Update Update the manual to reflect user-visible changes.
  • Type Labels Select a github label for the type: bugfix, enhancement, etc.
  • Product Labels Select a github label for the product: TaskVine, Makeflow, etc.
  • PR RTM Mark your PR as ready to merge.

…x to shared filesystem.

- Added shared_filesystem_link_mode to replace VINE_FILE with VINE_SHAREDFS.
- Rename load_from_shared_fs to shared_filesystem_cache_mode, which replaces VINE_FILE with VINE_URL.
@dthain dthain force-pushed the vine-shared-fs-mode branch from 1f2999c to cc66f42 Compare November 4, 2024 21:32
@dthain
Copy link
Member Author

dthain commented Nov 7, 2024

Ah, this is not going work to a variety of reasons, retrying again with a simpler approach.

@dthain dthain closed this Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant