Remove remote channel script directory handling #3688
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is only needed for channels executing on a remote system without a shared run directory with the submit side. This is no longer supported. See #3515.
khk-globus
approved these changes
Nov 7, 2024
benclifford
added a commit
that referenced
this pull request
Nov 8, 2024
This has been a basically dead code path since removal of non-remote channels: Either a user chooses to not move files (overriding move_files to False) and the push_file was not used. Or the default push_file was used, but: * this is always LocalChannel now (PR #3677) * the script directory is always the local script directory (PR #3688) and so the local channel code always skips making a copy This commit simplies all of that away into using the script path directly with putting it through the above complicated no-op.
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 26, 2024
This removes push/pull support from channels. All the HPC providers used channel.push_file to push their batch scripts to the (no-longer) remote system. This has been a basically dead code path since removal of non-remote channels: The channel is always a LocalChannel now (PR #3677) The "remote" script directory is always the local script directory (PR #3688) and so `LocalChannel.push_file` always skips making a copy and returns the path it was given without further action. So all the removed code is a no-op, and this PR simplifies that into nothing. # Changed Behaviour Some providers had an option to let users decide if scripts (and other files) would be pushed/pulled to the remote system. Those options are removed by this PR. ## Type of change - New feature - Code maintenance/cleanup
github-merge-queue bot
pushed a commit
that referenced
this pull request
Dec 3, 2024
This leaves channels as a stateless (and useless) object that can be removed in a subsequent PR. That removal will change/break a lot of config files so I want it to keep it separate from this attribute removal. # Changed Behaviour Batch providers will now use their own script_dir attribute, rather than the channel script_dir attribute. Prior to PR #3688 these attributes were paths on different file systems: the submit side (for providers) and the batch job execution side (for channels). PR #3688 removed support for batch job commands to run somewhere that isn't the workflow submit side, and so since then, these two attributes have been roughly equivalent. In some (obscure seeming cases) they might be different: if a channel is shared between DFKs, then the script_dir used by providers in one DFK will now no longer use the channel script dir set by the other DFK. This was probably a bug anyway but I am noting it because this PR isn't completely behaviour preserving. ## Type of change - Code maintenance/cleanup
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is only needed for channels executing on a remote system without a shared run directory with the submit side. This is no longer supported. See #3515.
Type of change