Skip to content

Commit

Permalink
Run jobs parallel again (#47)
Browse files Browse the repository at this point in the history
When running an ensemble, one of the changes in [Cleanup of requirements
and increase code
quality](45ef591),
made the jobs being run sequentially instead of parallel on the login
nodes. This MR reverts the relevant change.
  • Loading branch information
AnnikaLau authored Sep 19, 2024
1 parent a96b5b4 commit 798733b
Showing 1 changed file with 8 additions and 5 deletions.
13 changes: 8 additions & 5 deletions engine/run_ensemble.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,13 +109,16 @@ def prepare_perturbed_run_script(


def append_job(job, job_list, parallel):
with subprocess.Popen(job) as p:
if not parallel:
p.communicate()
p = subprocess.Popen(job) # pylint: disable=consider-using-with
if not parallel:
try:
time.sleep(5)
p.wait()
test_job_returncode(p)
else:
job_list.append(p)
finally:
p.kill()
else:
job_list.append(p)


def finalize_jobs(job_list, dry, parallel):
Expand Down

0 comments on commit 798733b

Please sign in to comment.