Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New SHARE and WORK variables pointing to the symlink'd location? #6488

Open
ColemanTom opened this issue Nov 22, 2024 · 1 comment
Open

New SHARE and WORK variables pointing to the symlink'd location? #6488

ColemanTom opened this issue Nov 22, 2024 · 1 comment
Labels
question Flag this as a question for the next Cylc project meeting.

Comments

@ColemanTom
Copy link
Contributor

Problem

In the global.cylc, we have our share and work directories on lustre, which are symlinked into our workflows area. Typically, we access those locations using things like CYLC_TASK_WORK_DIR and ROSE_DATAC - but they go through the symoblic link. Under low traffic, that is fine, but, we have shown under high traffic this can cause problems and it is less load on metadata servers to use the real path. Someone explained it to me a while ago, but I've forgotten what goes on each time a symbolic link is traversed.

Proposed Solution

When a workflow is installed, it obviously knows that real path as it sets the symlinks up. I would like those variables made available directly in a job script so we can avoid traversing the symbolic links when scripting. I think the symlinks are useful for navigating easily in interactive mode, but from an efficiency perspective, not using them inside a script is better.

@oliver-sanders
Copy link
Member

Suggest using realpath to fix these variables in the workflow definition like so:

[runtime]
    [[root]]
        [[[environment]]]]
            CYLC_TASK_WORK_DIR = $(realpath "$CYLC_TASK_WORK_DIR")

@oliver-sanders oliver-sanders added the question Flag this as a question for the next Cylc project meeting. label Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Flag this as a question for the next Cylc project meeting.
Projects
None yet
Development

No branches or pull requests

2 participants