Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable Radical/MPI test that was waiting on issue #3029 #3286

Merged
merged 13 commits into from
Jun 7, 2024

Conversation

benclifford
Copy link
Collaborator

The fix to this issue comes from a change in Radical Cybertools, not a change in the Parsl codebase.
See radical-cybertools/radical.saga#885

Fixes

Fixes #3029

Type of change

  • Code maintenance/cleanup

The fix to this issue comes from a change in Radical Cybertools, not a
change in the Parsl codebase.
See radical-cybertools/radical.saga#885
@benclifford
Copy link
Collaborator Author

setting this back to draft pending further info from Radical Cybertools team on how to get a more recent radical

@benclifford benclifford marked this pull request as draft March 24, 2024 12:22
@benclifford
Copy link
Collaborator Author

Revisiting this... I upgraded this to radical pilot 1.49. It still fails - on Python 3.12:

benc@clva:~/tmp/oe/.pytest/parsltest-current/runinfo/000/rpex.session.fv-az660-699.runner.019821.0000$ cat pmgr_launching.0000.log
1712577992.826 : pmgr_launching.0000  : 9604  : 139897463289664 : ERROR    : launcher PSI_J unusable
Traceback (most recent call last):
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/base.py", line 167, in __init__
    try   : raise e
            ^^^^^^^
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/base.py", line 156, in __init__
    self._launchers[name] = ctor(name, self._log, self._prof,
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/psi_j.py", line 40, in __init__
    raise psij_ex
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/psi_j.py", line 23, in <module>
    import psij
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/psij/__init__.py", line 13, in <module>
    from .job_executor import JobExecutor
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/psij/job_executor.py", line 3, in <module>
    from distutils.version import Version
ModuleNotFoundError: No module named 'distutils'
1712577992.828 : pmgr_launching.0000  : 9604  : 139897463289664 : ERROR    : launcher SAGA unusable
Traceback (most recent call last):
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/base.py", line 167, in __init__
    try   : raise e
            ^^^^^^^
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/base.py", line 156, in __init__
    self._launchers[name] = ctor(name, self._log, self._prof,
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/saga.py", line 35, in __init__
    raise rs_ex
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/saga.py", line 15, in <module>
    import radical.saga
ModuleNotFoundError: No module named 'radical.saga'
1712577995.888 : pmgr_launching.0000  : 9604  : 139897117595392 : ERROR    : bulk launch failed
Traceback (most recent call last):
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/base.py", line 306, in work
    self._start_pilot_bulk(resource, schema, pilots)
  File "/home/runner/work/parsl/parsl/.venv/lib/python3.12/site-packages/radical/pilot/pmgr/launching/base.py", line 524, in _start_pilot_bulk
    raise RuntimeError('no launcher found for %s' % pilot['uid'])
RuntimeError: no launcher found for pilot.0000

The second half of that is missing saga, which i guess is expected now? The first parsl looks like psi-j-python is expecting distutils - see https://peps.python.org/pep-0632/ for discussion about that including:

Code that imports distutils will no longer work from Python 3.12.

@benclifford benclifford marked this pull request as ready for review May 24, 2024 17:52
@benclifford benclifford requested a review from khk-globus June 7, 2024 12:04
@benclifford benclifford merged commit bc086f7 into master Jun 7, 2024
6 checks passed
@benclifford benclifford deleted the benc-radical-3029 branch June 7, 2024 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test failures from Radical pilot executor
2 participants