Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep info about output files within sandbox #3292

Open
mtitov opened this issue Dec 17, 2024 · 3 comments
Open

Keep info about output files within sandbox #3292

mtitov opened this issue Dec 17, 2024 · 3 comments

Comments

@mtitov
Copy link
Contributor

mtitov commented Dec 17, 2024

related to #3276
in src/radical/pilot/agent/executing/base.py

L288: https://github.com/radical-cybertools/radical.pilot/pull/3276/files#diff-f16b90015a62a91010ce901f4d97c31e25fc9c338945e13f82574013597cb6f7R288

L300: https://github.com/radical-cybertools/radical.pilot/pull/3276/files#diff-f16b90015a62a91010ce901f4d97c31e25fc9c338945e13f82574013597cb6f7R300

need to provide $RP_TASK_SANDBOX for .files and .ofiles files

from use case: pre-exec changes working directory and then that files appear there

@andre-merzky
Copy link
Member

The docstring specifically states:

        Warning:
            This can be incomplete: the heuristics will not detect files which
            start with `<task_id>.`, for example.  It will also not detect files
            which are not created in the task sandbox.

I would like to avoid this outside of the task sandbox: there is no telling if the task created the file, or if some other process or user did so. That is opening too many cans of wurms for my liking...

@mtitov
Copy link
Contributor Author

mtitov commented Dec 18, 2024

I mostly was about keeping RP related files within corresponding sandboxes, e.g., ... > $RP_TASK_SANDBOX/task.000000.files, otherwise this file (and .ofiles) appears outside of task sandbox

@andre-merzky
Copy link
Member

Ah, sorry, I misunderstood that. Thanks for clarifying!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants