Skip to content

Commit

Permalink
Test parsl+slurm in CI (Parsl#3606)
Browse files Browse the repository at this point in the history
  • Loading branch information
tylern4 authored and yadudoc committed Sep 19, 2024
1 parent a6003d2 commit 008e18b
Show file tree
Hide file tree
Showing 3 changed files with 77 additions and 0 deletions.
35 changes: 35 additions & 0 deletions .github/workflows/parsl+slurm.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Test Slurm Scheduler
on:
pull_request:

jobs:
build:
runs-on: ubuntu-22.04
permissions:
packages: read
strategy:
fail-fast: false
matrix:
container: ["ghcr.io/tylern4/slurm-standalone:slurm-24-05-0-1"]
timeout-minutes: 30

container:
image: ${{ matrix.container }}
options: "--platform=linux/amd64 --rm -h node01"

name: ${{ matrix.container }}
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Install Dependencies and Parsl
run: |
CC=/usr/lib64/openmpi/bin/mpicc pip3 install . -r test-requirements.txt
- name: Verify Parsl Installation
run: |
pytest parsl/tests/ -k "not cleannet and not unix_filesystem_permissions_required" --config parsl/tests/configs/local_threads.py --random-order --durations 10
- name: Test Parsl with Slurm Config
run: |
./parsl/tests/slurm-entrypoint.sh pytest parsl/tests/ -k "not cleannet and not unix_filesystem_permissions_required" --config parsl/tests/configs/slurm_local.py --random-order --durations 10
26 changes: 26 additions & 0 deletions parsl/tests/configs/slurm_local.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
from parsl.channels import LocalChannel
from parsl.config import Config
from parsl.executors import HighThroughputExecutor
from parsl.launchers import SrunLauncher
from parsl.providers import SlurmProvider


def fresh_config():
return Config(
executors=[
HighThroughputExecutor(
label="docker_slurm",
encrypted=True,
provider=SlurmProvider(
cmd_timeout=60, # Add extra time for slow scheduler responses
channel=LocalChannel(),
nodes_per_block=1,
init_blocks=1,
min_blocks=1,
max_blocks=1,
walltime='00:10:00',
launcher=SrunLauncher(),
),
)
],
)
16 changes: 16 additions & 0 deletions parsl/tests/slurm-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash
set -e

echo "---> Starting the MUNGE Authentication service (munged) ..."
gosu munge /usr/sbin/munged

echo "---> Starting the slurmctld ..."
exec gosu slurm /usr/sbin/slurmctld -i -Dvvv &

echo "---> Waiting for slurmctld to become active before starting slurmd..."

echo "---> Starting the Slurm Node Daemon (slurmd) ..."
exec /usr/sbin/slurmd -Dvvv &

echo "---> Running user command '${@}'"
exec "$@"

0 comments on commit 008e18b

Please sign in to comment.