Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transferring environment with condor jobs causes errors #222

Open
bistapf opened this issue Sep 19, 2024 · 1 comment
Open

Transferring environment with condor jobs causes errors #222

bistapf opened this issue Sep 19, 2024 · 1 comment

Comments

@bistapf
Copy link
Contributor

bistapf commented Sep 19, 2024

The job config scripts generated by this framework currently transfer the local environment to the batch machines, e.g. from send_lhep8.py here.

This causes issues with newer key4hep stack versions, which have an internal consistency check and will no longer execute the setup script when the respective environment variable is already set see code here.

First issue arises from unset commands also written to the job scripts from e.g. send_lhep8.py here, leaving the library paths missing since sourcing the key4hep stack again afterwards now does nothing.

Another issue is that the submitted jobs will then always use the key4hep release set in init.sh (so the latest), rather than the custom ones specified in the respective config/param (for newer releases, after the implementation of the internal check, in the past this is only worked because no such check was done by key4hep).

Suggested easiest solution would be to not transfer the environment and set up from scratch on the batch node, unless there are usecases where this will not work.

@bistapf
Copy link
Contributor Author

bistapf commented Sep 26, 2024

PR#218 will fix this for FCC-hh productions by not transferring the environment in this case, while the behaviour for FCC-ee productions will be unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant