Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create eval directory and batch directories with executable evaluator script in all Environments #90

Open
FlorianPommerening opened this issue Oct 27, 2023 · 0 comments

Comments

@FlorianPommerening
Copy link
Member

Currently, only the SlurmEnvironment writes states to disk. We want to unify this and use this setup for all cases (#53).
As part of this change, we also want to turn the evaluator script into an executable script (#82, this also fixes #79).
The evaluation can then use resource limits and react correctly to the evaluator running out of resources (left over from #72).
With one run_dir for each run, it is also easier to keep track and report errors that should be investigated but do not terminate the search (e.g., a critical error in an evaluator).
The plan is to leave all run directories on the disk after the search and logging behavior becomes easier (related to #57, #77).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant