Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/main'
Browse files Browse the repository at this point in the history
  • Loading branch information
jussienko committed Nov 25, 2024
2 parents 571572b + 44e61e1 commit ed2f838
Show file tree
Hide file tree
Showing 22 changed files with 667 additions and 232 deletions.
25 changes: 25 additions & 0 deletions .github/workflows/pages-html.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Deploy HTML slides to Pages

on:
# Runs on pushes targeting the default branch
push:
branches:
- "main"
paths:
- "docs/**"
- ".github/workflows/pages.yml"

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

jobs:
pages-html:
uses: ./.github/workflows/pages.yml
with:
include_pdf: false
17 changes: 17 additions & 0 deletions .github/workflows/pages-pdf.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Deploy HTML and PDF slides to Pages

on:
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

jobs:
pages-pdf:
uses: ./.github/workflows/pages.yml
with:
include_pdf: true
62 changes: 62 additions & 0 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Script based on examples in https://github.com/actions/starter-workflows/tree/main/pages
name: Deploy slides to Pages

on:
workflow_call:
inputs:
include_pdf:
required: true
type: boolean

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false

jobs:
build:
timeout-minutes: 30
runs-on: ubuntu-latest
container:
image: ghcr.io/csc-training/slidefactory:3.1.0-beta.6
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
id: pages
uses: actions/configure-pages@v4
- name: Build slides
env:
INCLUDE_PDF: ${{ inputs.include_pdf }}
run: |
git config --global --add safe.directory $PWD
GIT_SHORT_SHA=$(git rev-parse --short $GITHUB_SHA)
GIT_DATE=$(git show -s --format=%ci $GITHUB_SHA)
ARGS=""
[[ "$INCLUDE_PDF" == "true" ]] && ARGS="--with-pdf"
slidefactory pages about.yml build --info_content "Updated for [$GIT_SHORT_SHA]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/commit/$GITHUB_SHA) ($GIT_DATE)" $ARGS
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: ./build

deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ a.out
*.qdrep
*.sqlite
heat
build/
63 changes: 30 additions & 33 deletions Exercises_Instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,21 +38,21 @@ The Intel DevCloud can be acces via the [web interface](https://console.cloud.in

### Disk area

The (computing and storage) resources can be accessed on on supercomputers via project-based allocation system, where users are granted access based on the specific needs and goals of their projects. Running applications and storage area are directly linked ot this projects. For this event we have been granted access to the training `project_2008874` on Mahti and `project_462000456` on LUMI.
The (computing and storage) resources can be accessed on on supercomputers via project-based allocation system, where users are granted access based on the specific needs and goals of their projects. Running applications and storage area are directly linked ot this projects. For this event we have been granted access to the training `project_2012125` on Mahti and `project_462000456` on LUMI.

All the exercises in the supercomputers have to be carried out in the **scratch** disk area. The name of the scratch directory can be queried with the commands `csc-workspaces` on Mahti and `lumi-workspaces` onLUMI. As the base directory is shared between members of the project, you should create your own
directory:

on Mahti
```
cd /scratch/project_2008874
cd /scratch/project_2012125
mkdir -p $USER
cd $USER
```

on LUMI
```
cd /scratch/project_462000456
cd /scratch/project_462000752
mkdir -p $USER
cd $USER
```
Expand Down Expand Up @@ -122,19 +122,18 @@ In order to use the intel SYCL compiler one has to set the environment varibles

on Mahti:
```
. /projappl/project_2008874/intel/oneapi/setvars.sh --include-intel-llvm
. /projappl/project_2012125/intel/oneapi/setvars.sh --include-intel-llvm
module load cuda # This is needed for compiling sycl code for nvidia gpus
module load openmpi/4.1.2-cuda # This is neeeded for using CUDA aware MPI
```

on LUMI:
```
. /projappl/project_462000456/intel/oneapi/setvars.sh --include-intel-llvm
. /projappl/project_462000752/intel/oneapi/setvars.sh --include-intel-llvm
module load LUMI/22.08
module load LUMI
module load partition/G
module load rocm/5.3.3
module load cce/16.0.1
module load rocm/6.0.3
export MPICH_GPU_SUPPORT_ENABLED=1 # Needed for GPU aware MPI
```
After this one can load other modules that might be needed for compiling the codes. With the environment set-up we can compile and run the SYCL codes.
Expand All @@ -158,21 +157,20 @@ module load cuda # This is needed for compiling sycl code for nvidia gpus
module load openmpi/4.1.2-cuda # This is neeeded for using CUDA aware MPI
```
```
/projappl/project_2008874/AdaptiveCpp/bin/acpp -fuse-ld=lld -O3 -L/appl/spack/v017/install-tree/gcc-8.5.0/gcc-11.2.0-zshp2k/lib64 <sycl_code>.cpp
/projappl/project_2012125/AdaptiveCpp/bin/acpp -fuse-ld=lld -O3 -L/appl/spack/v017/install-tree/gcc-8.5.0/gcc-11.2.0-zshp2k/lib64 <sycl_code>.cpp
```
on LUMI:
```
module load LUMI/22.08
module load LUMI
module load partition/G
module load rocm/5.3.3
module load cce/16.0.1
module load rocm/6.0.3
export MPICH_GPU_SUPPORT_ENABLED=1
export LD_LIBRARY_PATH=/appl/lumi/SW/LUMI-22.08/G/EB/Boost/1.79.0-cpeCray-22.08/lib:$LD_LIBRARY_PATH
export LD_PRELOAD=/pfs/lustrep4/appl/lumi/SW/LUMI-22.08/G/EB/rocm/5.3.3/llvm/lib/libomp.so
#export LD_LIBRARY_PATH=/appl/lumi/SW/LUMI-22.08/G/EB/Boost/1.79.0-cpeCray-22.08/lib:$LD_LIBRARY_PATH ???
#export LD_PRELOAD=/pfs/lustrep4/appl/lumi/SW/LUMI-22.08/G/EB/rocm/5.3.3/llvm/lib/libomp.so ??????
```

```
/projappl/project_462000456/AdaptiveCpp/bin/acpp -O3 <sycl_code>.cpp
/projappl/project_462000752/AdaptiveCpp/bin/acpp -O3 <sycl_code>.cpp
```
In general one can set specific targets via the `--acpp-targets` flag, but we set-up AdaptiveCpp so that on Mahti the `acpp` compiler will automatically generate code for CPU and Nvidia GPUs, while on LUMI for CPU and AMD GPUs.

Expand All @@ -198,17 +196,16 @@ icpx -fuse-ld=lld -fsycl -fsycl-targets=nvptx64-nvidia-cuda -Xsycl-target-backen
```
or
```
/projappl/project_2008874/AdaptiveCpp/bin/acpp -fuse-ld=lld -O3 -L/appl/spack/v017/install-tree/gcc-8.5.0/gcc-11.2.0-zshp2k/lib64 `mpicxx --showme:compile` `mpicxx --showme:link` <sycl_mpi_code>.cpp
/projappl/project_2012125/AdaptiveCpp/bin/acpp -fuse-ld=lld -O3 -L/appl/spack/v017/install-tree/gcc-8.5.0/gcc-11.2.0-zshp2k/lib64 `mpicxx --showme:compile` `mpicxx --showme:link` <sycl_mpi_code>.cpp
```

Similarly on LUMI. First we set up the envinronment and load the modules as indicated above
```
. /projappl/project_462000456/intel/oneapi/setvars.sh --include-intel-llvm
. /projappl/project_462000752/intel/oneapi/setvars.sh --include-intel-llvm
module load LUMI/22.08
module load LUMI
module load partition/G
module load rocm/5.3.3
module load cce/16.0.1
module load rocm/6.0.3
export MPICH_GPU_SUPPORT_ENABLED=1
```
Now compile with intel compilers:
Expand All @@ -218,8 +215,8 @@ icpx -fsycl -fsycl-targets=amdgcn-amd-amdhsa,spir64_x86_64 -Xsycl-target-backend
```
Or with AdaptiveCpp:
```
export LD_PRELOAD=/pfs/lustrep4/appl/lumi/SW/LUMI-22.08/G/EB/rocm/5.3.3/llvm/lib/libomp.so
/projappl/project_462000456/AdaptiveCpp/bin/acpp -O3 `CC --cray-print-opts=cflags` <sycl_mpi_code>.cpp `CC --cray-print-opts=libs`
#export LD_PRELOAD=/pfs/lustrep4/appl/lumi/SW/LUMI-22.08/G/EB/rocm/5.3.3/llvm/lib/libomp.so
/projappl/project_462000752/AdaptiveCpp/bin/acpp -O3 `CC --cray-print-opts=cflags` <sycl_mpi_code>.cpp `CC --cray-print-opts=libs`
```

## Running applications in supercomputers
Expand All @@ -231,9 +228,9 @@ The `job.sh` file contains all the necessary information (number of nodes, tasks

### Useful environment variables

Use [`SYCL_PI_TRACE`](https://intel.github.io/llvm-docs/EnvironmentVariables.html#sycl-pi-trace-options) to enable runtime tracing (e.g. device discovery):
Use [`SYCL_UR_TRACE`](https://intel.github.io/llvm-docs/EnvironmentVariables.html#sycl-pi-trace-options) to enable runtime tracing (e.g. device discovery):

export SYCL_PI_TRACE=1
export SYCL_UR_TRACE=1


### Running on Mahti
Expand All @@ -242,7 +239,7 @@ Use [`SYCL_PI_TRACE`](https://intel.github.io/llvm-docs/EnvironmentVariables.htm
```
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=project_2008874
#SBATCH --account=project_2012125
#SBATCH --partition=medium
#SBATCH --reservation=hlgp-cpu-f2024
#SBATCH --time=00:05:00
Expand All @@ -258,7 +255,7 @@ The output of job will be in file `slurm-xxxxx.out`. You can check the status of
`scancel JOBID`.

The reservation `hlgp-cpu-f2024` for partition `medium` is available during the training days and it
is accessible only if the users are part of `project_2008874`.
is accessible only if the users are part of `project_2012125`.

Some applications use MPI, in this case the number of node and number of tasks per node will have to be adjusted accordingly.

Expand All @@ -270,9 +267,9 @@ single GPU with single MPI task and a single thread use:
```
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=project_2008874
#SBATCH --account=project_2012125
#SBATCH --partition=gpusmall
#SBATCH --reservation=hlgp-gpu-f2024-thu
#SBATCH --reservation=hlgp-gpu-f2024-thu ?????????
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:05:00
Expand All @@ -282,7 +279,7 @@ srun my_gpu_exe
```
The reservation `hlgp-gpu-f2024-wed` is valid on Wednesday, 15:00 to 17:00. On Thursday we will use `hlgp-gpu-f2024-thu` , while on Friday `hlgp-gpu-f2024-fri`. Outside the course hours, you can use gputest partition instead without the reservation argument, ie,
```
srun --account=project_2008874 --nodes=1 --partition=gputest --gres=gpu:a100:1 --time=00:05:00 ./my_gpu_exe
srun --account=project_2012125 --nodes=1 --partition=gputest --gres=gpu:a100:1 --time=00:05:00 ./my_gpu_exe
```


Expand All @@ -295,9 +292,9 @@ LUMI is similar to Mahti.
```
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=project_462000456
#SBATCH --account=project_462000752
#SBATCH --partition=standard
##SBATCH --reservation=hlgp-cpu-f2024 # The reservation does not work
##SBATCH --reservation=hlgp-cpu-f2024 ??????# The reservation does not work
#SBATCH --time=00:05:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
Expand All @@ -312,9 +309,9 @@ srun my_cpu_exe
```
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=project_462000456
#SBATCH --account=project_462000752
#SBATCH --partition=standard-g
#SBATCH --reservation=hlgp-gpu-f2024
#SBATCH --reservation=hlgp-gpu-f2024 ??????
#SBATCH --time=00:05:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
Expand Down
File renamed without changes.
Loading

0 comments on commit ed2f838

Please sign in to comment.