ImpactX: the next generation of the IMPACT-Z code
In order to learn how to install and run the code, please see the online documentation: https://impactx.readthedocs.io
- ImpactX Doxygen: https://impactx.readthedocs.io/en/latest/_static/doxyhtml
- AMReX Doxygen: https://amrex-codes.github.io/amrex/doxygen
- WarpX Doxygen: https://warpx.readthedocs.io/en/latest/_static/doxyhtml
Our workflow is described in CONTRIBUTING.rst.
Please prepare you local development environment as follows. Pick one of the methods below:
ssh perlmutter-p1.nersc.govmodule load cmake/git-20210830 # 3.22-dev
module swap PrgEnv-nvidia PrgEnv-gnu
module swap gcc gcc/9.3.0
module load cuda
module load cray-hdf5-parallel/1.12.0.7
# GPU-aware MPI
export MPICH_GPU_SUPPORT_ENABLED=1
# optimize CUDA compilation for A100
export AMREX_CUDA_ARCH=8.0
# compiler environment hints
export CC=$(which gcc)
export CXX=$(which g++)
export FC=$(which gfortran)
export CUDACXX=$(which nvcc)
export CUDAHOSTCXX=$(which g++)# configure
cmake -S . -B build_perlmutter -DImpactX_COMPUTE=CUDA
# compile
cmake --build build_perlmutter -j 10
# run
cd build_perlmutter/bin
srun -N 1 --ntasks-per-node=4 -t 0:10:00 -C gpu -c 32 -G 4 --qos=debug -A m3906_g ./impactx ../../examples/input_fodo.inssh cori.nersc.govmodule swap craype-haswell craype-mic-knl
module swap PrgEnv-intel PrgEnv-gnu
module load cmake/3.21.3
module load cray-hdf5-parallel/1.10.5.2
module load cray-fftw/3.3.8.4
module load cray-python/3.7.3.2# configure
cmake -S . -B build_cori
# compile
cmake --build build_cori -j 8
# run
cd build_cori/bin
srun -C knl -N 1 -t 30 -q debug ./impactx ../../examples/input_fodo.inbrew update
brew install adios2 # for openPMD
brew install ccache
brew install cmake
brew install fftw
brew install git
brew install hdf5-mpi # for openPMD
brew install libomp # for OpenMP
brew install pkg-config # for fftw
brew install open-mpisudo apt update
sudo apt install build-essential ccache cmake g++ git libfftw3-mpi-dev libfftw3-dev libhdf5-openmpi-dev libopenmpi-dev pkg-config python3 python3-matplotlib python3-numpy python3-scipyspack env create impactx-dev
spack env activate impactx-dev
spack add adios2 # for openPMD
spack add ccache
spack add cmake
spack add fftw
spack add hdf5 # for openPMD
spack add mpi
spack add pkgconfig # for fftw
# OpenMP support on macOS
[[ $OSTYPE == 'darwin'* ]] && spack add llvm-openmp
# optional:
# spack add cuda
# spack add python
# spack add py-pip
spack install(in new terminals, re-activate the environment with spack env activate impactx-dev again)
conda create -n impactx-dev -c conda-forge adios2 ccache cmake compilers git hdf5 fftw matplotlib ninja
conda activate impactx-dev
# compile with -DImpactX_MPI=OFFBefore you start, you will need a copy of the ImpactX source code:
git clone [email protected]:ECP-WarpX/impactx.git
cd impactx# find dependencies & configure
cmake -S . -B build
# compile
cmake --build build -j 4That's all!
ImpactX binaries are now in build/bin/.
Most people execute these binaries directly or copy them out.
You can inspect and modify build options after running cmake -S . -B build with either
ccmake buildor by adding arguments with -D<OPTION>=<VALUE> to the first CMake call, e.g.:
cmake -S . -B build -DImpactX_COMPUTE=CUDA -DImpactX_MPI=OFFAn executable ImpactX binary with the current compile-time options encoded in its file name will be created in build/bin/.
Additionally, a symbolic link named impactx can be found in that directory, which points to the last built ImpactX executable.
The command-line syntax for this executable is:
Usage: impactx <inputs-file> [some.overwritten.option=value]...
Mandatory arguments (remove the <>):
inputs-file the path to an input file; can be relative to the current
working directory or absolute.
Example: input_fodo.in
Optional arguments (remove the []):
options this can overwrite any line in an inputs-file
Example: quad1.ds=0.5 sbend1.rc=1.5
Examples:
In the current working directory, there is a file "input_fodo.in" and the
"impactx" executable.
The line to execute would look like this:
./impactx input_fodo.in
In the current working directory, there is a file "input_fodo.in" and the
executable "impactx" is in a directory that is listed in the "PATH"
environment variable.
The line to execute would look like this:
impactx input_fodo.in
In the current working directory, there is a file "input_fodo.in" and the
"impactx" executable. We want to voerwrite the segment length of the beamline
element "quad1" that is already defined in it. We also want to change the
radius of curvature of the bending magnet "sbend1" to a different value than
in the file "input_fodo.in".
The line to execute would look like this:
./impactx input_fodo.in quad1.ds=0.5 sbend1.rc=1.5This work was supported by the Laboratory Directed Research and Development Program of Lawrence Berkeley National Laboratory under U.S. Department of Energy Contract No. DE-AC02-05CH11231.
Copyright (c) 2021, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved.
If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Intellectual Property Office at [email protected].
This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit others to do so.