GPU configuration

[lucaparisi91]

We do not have support in spack for GPU. This will likely involve modifying the package.yaml file.

Target would be to install a GPU package and checking that the package works and performs OK.

[lucaparisi91]

An initial configuration use by Nathan for Namd:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs:
  - quantum-espresso%gcc
  - cray-fftw
  - cray-mpich
  - cray-libsci
  - [email protected]%gcc fftw=cray-fftw ^charmpp+smp backend=mpi build-target=charm++ ^tcl+pthreads
  - [email protected]%gcc fftw=cray-fftw ^charmpp~smp backend=mpi build-target=charm++ ^tcl+pthreads
  - [email protected]%gcc fftw=cray-fftw ^charmpp+smp~shared backend=mpi build-target=charm++ ^tcl+pthreads
  - [email protected]%gcc fftw=cray-fftw ^charmpp~smp~shared backend=mpi build-target=charm++ ^tcl+pthreads
  - [email protected]%gcc+rocm amdgpu_target=gfx90a fftw=cray-fftw ^charmpp+smp backend=mpi build-target=charm++ ^tcl+pthreads
  # - [email protected]%gcc+rocm amdgpu_target=gfx90a fftw=cray-fftw ^charmpp+smp~shared backend=mpi build-target=charm++ pmi=cray-pmi ^tcl+pthreads
  concretizer:
    unify: when_possible
  modules:
    view_relative_modules:
      use_view: full
    default:
      roots:
        lmod: $SPACK_ENV/modules
      arch_folder: false
      enable:
      - lmod
      lmod:
        hide_implicits: true
        hash_length: 0
        projections:
          all: '{name}/{version}'
          namd^charmpp~smp: '{name}-nosmp/{version}'
          namd+rocm: '{name}-gpu/{version}'
        all:
          autoload: direct
        namd:
          environment:
            set:
              SLURM_CPU_FREQ_REQ: '2250000'
  view:
    # default:
    #   root: $SPACK_ENV/views/default
    #   projections:
    #     all: '{name}-{version}-{hash}'
    full:
      root: $SPACK_ENV/views/full
      projections:
        all: '{compiler.name}-{compiler.version}/{name}-{version}'
        namd^charmpp~smp: '{compiler.name}-{compiler.version}/{name}-{version}-nosmp'
        namd+rocm: '{compiler.name}-{compiler.version}/{name}-{version}-gpu'
        charmpp~smp: '{compiler.name}-{compiler.version}/{name}-{version}-nosmp'
      # link_type: symlink
Show more
If its useful, this is my additional config for adding GPU support (at least for NAMD). It was added in packages.yaml in my personal spack config:

packages:
  cray-mpich:
    externals:
    - spec: [email protected]%gcc
      modules:
      - cray-mpich/8.1.23
      - craype-network-ofi
      prefix: /opt/cray/pe/mpich/8.1.23/ofi/gnu/9.1

    - spec: [email protected]%aocc
      modules:
      - cray-mpich/8.1.23
      - craype-network-ofi
      prefix: /opt/cray/pe/mpich/8.1.23/ofi/aocc/3.0
    - spec: [email protected]%cce
      modules:
      - cray-mpich/8.1.23
      - craype-network-ofi
      prefix: /opt/cray/pe/mpich/8.1.23/ofi/cray/10.0
    - spec: [email protected]%rocmcc
      modules:
      - cray-mpich/8.1.23
      - craype-network-ofi
      prefix: /opt/cray/pe/mpich/8.1.23/ofi/rocm-compiler/5.0
    buildable: false

  hip:
    externals:
    - spec: [email protected]
      modules:
      - rocm/5.2.3
      prefix: /opt/rocm-5.2.3/hip
    buildable: false

  hsa-rocr-dev:
    externals:
    - spec: [email protected]
      modules:
      - rocm/5.2.3
      prefix: /opt/rocm-5.2.3/hsa
    buildable: false

  llvm-amdgpu:
    externals:
    - spec: [email protected]
      modules:
      - rocm/5.2.3
      prefix: /opt/rocm-5.2.3/llvm
    buildable: false

#  cray-pmi:
#    externals:
#    - spec: [email protected]
#      modules:
#      - cpe/22.12
#      - PrgEnv-gnu
#      - cray-pmi/6.1.8
#      - cray-fftw
#      prefix: /opt/cray/pe/pmi/6.1.8
#      extra_attributes:
#        environment:
#          set:
#            VERBOSE: 1
#    buildable: false

[lucaparisi91]

Also see a configuration on Frontier at https://github.com/E4S-Project/facility-external-spack-configs/tree/master/frontier .
Should be very similar to Archer2.