Fedora compiler environment

Supplementary instructions for Fedora Linux environments

Noted differences between Fedora Core (FC) 21 and Fedora Core 23. Useful commands to find the software we need to compile MOM6.

Package management: FC21: yum, FC23: dnf
List files in a package: FC21: rpm -ql [pkg], FC23: dnf repoquery -l [pkg]

These instructions closely follow the instructions outlined for GFDL's HPC platforms.

Setup a compiler environment on Fedora

Execute the following commands to install the compilation prerequisites on an Fedora machine:

sudo dnf install csh
sudo dnf install make
sudo dnf install gcc-gfortran
sudo dnf install openmpi
sudo dnf install openmpi-devel
sudo dnf install netcdf
sudo dnf install netcdf-devel
sudo dnf install netcdf-fortran
sudo dnf install netcdf-fortran-devel
sudo dnf install netcdf-fortran-openmpi
sudo dnf install netcdf-fortran-openmpi-devel

Other recommended tools to use git and X windows via SSH:

sudo dnf install git
sudo dnf install xauth
sudo dnf install xterm

Install the MOM6 source tree

You are ready to use git to download and setup the code repository on the AWS instance.

Create a blank `env` file

The following commands make use of a file build/gnu/env that can contain any local environment changes need for compilation:

mkdir -p build/gnu
echo > build/gnu/env

Example contents of env

# Need openmpi path before all other paths to get correct nc-config for netcdf with openmpi
export PATH=/usr/lib64/openmpi/bin:${PATH}
export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:/usr/lib64/openmpi/lib/pkgconfig

NOTE: It was necessary to perform export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:/usr/lib64/openmpi/lib/pkgconfig on the command line before compiling was successful. Sourcing on the fly as used below does not seem to work.

Create a copy of the linux-gnu template and modify

cd build/mkmf/templates
cp linux-gnu.mk linux-gnu-fedora23.mk
# Since we are using openmpi, change references of mpich2-* to ompi-*
# in linux-gnu-fedora23.mk

Building libfms.a

To build the Makefile use:

mkdir -p build/gnu/shared/repro/
(cd build/gnu/shared/repro/; rm -f path_names; \
../../../mkmf/bin/list_paths ../../../../src/FMS; \
../../../mkmf/bin/mkmf -t ../../../mkmf/templates/linux-gnu-fedora23.mk -p libfms.a -c "-Duse_libMPI -Duse_netCDF -DSPMD" path_names)

To build the FMS library (libfms.a):

(cd build/gnu/shared/repro/; source ../../env; make NETCDF=3 REPRO=1 FC=mpif90 CC=mpicc libfms.a -j)

Notice the addition of FC=mpif90 and CC=mpicc above, which is necessary to find the MPI header files.

Building the ocean-only MOM6 executable

To create the Makefile for compiling MOM6:

mkdir -p build/gnu/ocean_only/repro/
(cd build/gnu/ocean_only/repro/; rm -f path_names; \
../../../mkmf/bin/list_paths ./ ../../../../src/MOM6/{config_src/dynamic,config_src/solo_driver,src/{*,*/*}}/ ; \
../../../mkmf/bin/mkmf -t ../../../mkmf/templates/linux-gnu.mk -o '-I../../shared/repro' -p 'MOM6 -L../../shared/repro -lfms' -c "-Duse_libMPI -Duse_netCDF -DSPMD" path_names)

To build the MOM6 executable:

(cd build/gnu/ocean_only/repro/; source ../../env; make NETCDF=3 REPRO=1 FC=mpif90 LD=mpif90 MOM6 -j)

This time note the addition of LD=mpif90.

Run MOM6 ocean-only

For any example in ocean_only/ that does NOT have an INPUT/ directory, e.g. double_gyre:

cd ocean_only/double_gyre/
mkdir RESTART
(source ./../../build/gnu/env ; mpirun -np 4 ../../build/gnu/ocean_only/repro/MOM6)

Supplemental instructions on how to start a single instance on the Amazon Web Services (AWS) Elastic Compute Cloud (EC2)

For Fedora Core 23 (FC23), several pre-constructed base images are available from the Community AMIs. The ami name will be different for different compute regions and zones. By searching on "Fedora-Cloud-Base-23" you can see all the different zones and types available. Linux Amazon Machine Images use one of two types of virtualization: paravirtual (PV) or hardware virtual machine (HVM). The main difference between PV and HVM AMIs is the way in which they boot and whether they can take advantage of special hardware extensions (CPU, network, and storage) for better performance. For example, the image ami-20203441 is a HVM image which can take advantage of the special hardware extensions.

Reference: Nic Hannah, http://nh.id.au/run-an-ocean-model-in-30-mins.html