Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(WIP) update centos7 installer #1110

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

(WIP) update centos7 installer #1110

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
425633c
improve mpi4py error message and update centos7 installer
d-a-v Jan 28, 2020
68846ea
more cmdline options to separate deps installations, compilation-inst…
d-a-v Jan 29, 2020
8e5f93c
remove depedency on lsb_release
d-a-v Jan 29, 2020
47819e7
verbosity in test script
d-a-v Jan 29, 2020
2e10a65
add options for building in docker
d-a-v Mar 2, 2020
f4d5469
Merge branch 'master' into temp
d-a-v Mar 2, 2020
39d8a3a
fixes per review, fixes for ubuntu 16.04
d-a-v Mar 2, 2020
bd23239
Merge branch 'master' into fixCentos7Installer
d-a-v Mar 5, 2020
63b4080
Merge branch 'master' into fixCentos7Installer
d-a-v Apr 14, 2020
5824594
Merge branch 'master' into fixCentos7Installer
d-a-v May 13, 2020
57c3c12
fix MPI builds automation for ubuntu and centos
d-a-v May 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 80 additions & 17 deletions contrib/build-meep.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ EOF

[ -z "$1" ] && echo "(use -h for help)"

test=false
installdeps=true

while [ ! -z "$1" ]; do
Expand All @@ -35,6 +36,13 @@ while [ ! -z "$1" ]; do
-n) # do not check for distribution dependencies
installdeps=false
;;
-S) # source directory (default: <installdir>/src)
SRCDIR="$2"
shift
;;
-t) # basic test
test=true
;;
*)
echo "'$1' ?"
help "$0"
Expand All @@ -45,16 +53,17 @@ done


# detect wether DESTDIR is ending with src/
[ -z ${DESTDIR} ] && DESTDIR=$(pwd)
[ ${DESTDIR##*/} = src ] && DESTDIR=$(cd $(pwd)/..; pwd)
SRCDIR=${DESTDIR}/src
[ -z "${DESTDIR}" ] && DESTDIR=$(pwd)
[ "${DESTDIR##*/}" = src ] && DESTDIR=$(cd $(pwd)/..; pwd)
[ -z "$SRCDIR" ] && SRCDIR=${DESTDIR}/src

cat << EOF

This sript will download or update sources, compile and install MEEP.
Please ensure the following final paths fit your needs:
Please ensure the following final paths fit your needs
Or use help '$0 -h' to change them:
'${DESTDIR}/bin/meep'
'${DESTDIR}/lib/...'
'${DESTDIR}/lib(64)/...'
'${DESTDIR}/share/...'
'${DESTDIR}/...'
'${SRCDIR}/<sources>'
Expand All @@ -63,6 +72,28 @@ Press return to continue
EOF
read junk

if $ubuntu; then
RPATH_FLAGS="-Wl,-rpath,${DESTDIR}/lib:/usr/lib/x86_64-linux-gnu/hdf5/openmpi"
LDFLAGS="-L${DESTDIR}/lib -L/usr/lib/x86_64-linux-gnu/hdf5/openmpi ${RPATH_FLAGS}"
CFLAGS="-I${DESTDIR}/include -I/usr/include/hdf5/openmpi"
fi

if $centos; then
export CC=/usr/lib64/openmpi/bin/mpicc
export CXX=/usr/lib64/openmpi/bin/mpicxx
export PATH=${PATH}:/usr/lib64/openmpi/bin
d-a-v marked this conversation as resolved.
Show resolved Hide resolved
RPATH_FLAGS="-Wl,-rpath,${DESTDIR}/lib64:/usr/lib64/openmpi/lib"
LDFLAGS="-L${DESTDIR}/lib64 -L/usr/lib64/openmpi/lib ${RPATH_FLAGS}"
CFLAGS="-I${DESTDIR}/include -I/usr/include/openmpi-x86_64/"
export PYTHONPATH=${DESTDIR}/lib/python3.6/site-packages
export PYTHONPATH=${PYTHONPATH}:/usr/local/lib64/python3.6/site-packages
export PYTHONPATH=${PYTHONPATH}:/usr/local/lib64/python3.6/site-packages/mpi4py
export LD_LIBRARY_PATH=${DESTDIR}/lib64:/usr/lib64/openmpi/lib:/usr/local/lib64/python3.6/site-packages/mpi4py/lib-pmpi:/usr/local/lib64/python3.6/site-packages/mpi4py
d-a-v marked this conversation as resolved.
Show resolved Hide resolved
export LD_PRELOAD=/usr/lib64/openmpi/lib/libmpi.so
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is preloading needed?

Copy link
Contributor Author

@d-a-v d-a-v Mar 2, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is preloading needed?

I don't know the reason TBH but I can show the effect of not using it with centos 7.

If I'm missing something I don't know yet where. This is not needed for ubuntu.

[root@771c413e32e4 /]# python3 /tmp/test-meep.py 

**
** failed to load python MPI module (mpi4py)
** /usr/local/lib64/python3.6/site-packages/mpi4py/MPI.cpython-36m-x86_64-linux-gnu.so: undefined symbol: ompi_mpi_logical8 
**

Vector3<16.0, 8.0, 0.0>

Elapsed run time = 0.0002 s
[root@771c413e32e4 /]# export LD_PRELOAD=/usr/lib64/openmpi/lib/libmpi.so
[root@771c413e32e4 /]# python3 /tmp/test-meep.py 
Using MPI version 3.0, 1 processes
Vector3<16.0, 8.0, 0.0>

Elapsed run time = 0.0002 s
[root@771c413e32e4 /]#

(It is a docker build <= me trying to explain 'root' :)

edit: details of this error message is visible thanks to #1140 (already applied in this PR)

fi

if ! $test; then

if ! lsb_release; then
echo "Minimum requirements:"
echo " Ubuntu:"
Expand Down Expand Up @@ -116,8 +147,11 @@ gitclone ()

autogensh ()
{
sh autogen.sh PKG_CONFIG_PATH="${PKG_CONFIG_PATH}" RPATH_FLAGS="${RPATH_FLAGS}" LDFLAGS="${LDFLAGS}" CFLAGS="${CFLAGS}" CPPFLAGS="${CPPFLAGS}" \
--disable-static --enable-shared --prefix="${DESTDIR}" \
LIB64="${DESTDIR}/lib"
$centos && LIB64="${DESTDIR}/lib64"
LLP="${LD_LIBRARY_PATH}:${LIB64}"
sh autogen.sh PKG_CONFIG_PATH="${PKG_CONFIG_PATH}" RPATH_FLAGS="${RPATH_FLAGS}" LDFLAGS="${LDFLAGS}" CFLAGS="${CFLAGS}" CPPFLAGS="${CPPFLAGS}" LD_LIBRARY_PATH=${LLP} \
--disable-static --enable-shared --prefix="${DESTDIR}" --libdir=${LIB64} \
--with-libctl=${DESTDIR}/share/libctl \
"$@"
}
Expand Down Expand Up @@ -152,12 +186,7 @@ if $installdeps && $ubuntu; then
sudo -H pip3 install --no-cache-dir mpi4py
export HDF5_MPI="ON"
sudo -H pip3 install --no-binary=h5py h5py
sudo -H pip3 install matplotlib>3.0.0

RPATH_FLAGS="-Wl,-rpath,${DESTDIR}/lib:/usr/lib/x86_64-linux-gnu/hdf5/openmpi"
LDFLAGS="-L${DESTDIR}/lib -L/usr/lib/x86_64-linux-gnu/hdf5/openmpi ${RPATH_FLAGS}"
CFLAGS="-I${DESTDIR}/include -I/usr/include/hdf5/openmpi"

sudo -H pip3 install matplotlib\>3.0.0
fi

if $installdeps && $centos; then
Expand Down Expand Up @@ -190,6 +219,12 @@ if $installdeps && $centos; then
systemtap \
wget

sudo yum -y install \
python3 \
python3-devel \
python36-numpy \
python36-scipy

sudo yum -y install \
openblas-devel \
fftw3-devel \
Expand All @@ -213,10 +248,7 @@ if $installdeps && $centos; then
guile-devel \
swig

export PATH=${PATH}:/usr/lib64/openmpi/bin
RPATH_FLAGS="-Wl,-rpath,${DESTDIR}/lib:/usr/lib64/openmpi/lib"
LDFLAGS="-L${DESTDIR}/lib -L/usr/lib64/openmpi/lib ${RPATH_FLAGS}"
CFLAGS="-I${DESTDIR}/include -I/usr/include/openmpi-x86_64/"
sudo -E pip3 install mpi4py
fi

CPPFLAGS=${CFLAGS}
Expand Down Expand Up @@ -263,3 +295,34 @@ autogensh --with-mpi --with-openmp PYTHON=python3
make -j && $SUDO make install

# all done

if $centos; then
cd ${DESTDIR}/lib/python3.6/site-packages/meep/
for i in ../../../../lib64/python3.6/site-packages/meep/*meep*; do
ln -sf $i
done
fi

fi # ! $test

########
# test

test=/tmp/test-meep.py

echo "export PYTHONPATH=${PYTHONPATH}"
echo "export PATH=${PATH}"
echo "export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}"
echo "export LD_PRELOAD=${LD_PRELOAD}"
cat << EOF > $test
import meep as mp
cell = mp.Vector3(16,8,0)
exit()
EOF

set -x
cat $test
python3 $test
set +x

########
4 changes: 2 additions & 2 deletions python/meep.i
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1563,8 +1563,8 @@ PyObject *_get_array_slice_dimensions(meep::fields *f, const meep::volume &where
if with_mpi():
try:
from mpi4py import MPI
except ImportError:
print('\n**\n** failed to load python MPI module (mpi4py)\n**\n')
except ImportError as e:
print('\n**\n** failed to load python MPI module (mpi4py)\n**', e, '\n**\n')
d-a-v marked this conversation as resolved.
Show resolved Hide resolved
pass
else:
# this variable reference is needed for lazy initialization of MPI
Expand Down