lustreapi.h is not useable

[Tristan-Kosciuch]

Thank you for taking the time to submit an issue!

Background information

What version of Open MPI are you using? (e.g., v3.0.5, v4.0.2, git branch name and hash, etc.)

4.1.4 downloaded from https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-4.1.4.tar.gz

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

curl -o openmpi-4.1.4.tar.gz https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-4.1.4.tar.gz
gunzip -c openmpi-4.1.4.tar.gz | tar xf -
cd openmpi-4.1.4
./configure --prefix=${MODINSTALLPATH}/ompi/4.1.4 --enable-mpi-cxx --with-lustre=/usr/src/lustre-client-modules-2.14.0-ddn54/lustre/
make all install

Please describe the system on which you are running

• Operating system/version: Ubuntu 20.04
• Computer hardware:
• Network type:

---

Details of the problem

When running the /configure step, the configure fails to use the lustreapi.h I am using the version of Lustre provided by DDN for use on GCP.

--- MCA component fs:lustre (m4 configuration macro)
checking for MCA component fs:lustre compile mode... dso
checking --with-lustre value... sanity check ok (/usr/src/lustre-client-modules-2.14.0-ddn54/lustre/)
checking looking for lustre libraries and header files in... (/usr/src/lustre-client-modules-2.14.0-ddn54/lustre/)
checking lustre/lustreapi.h usability... no
checking lustre/lustreapi.h presence... yes
configure: WARNING: lustre/lustreapi.h: present but cannot be compiled
configure: WARNING: lustre/lustreapi.h:     check for missing prerequisite headers?
configure: WARNING: lustre/lustreapi.h: see the Autoconf documentation
configure: WARNING: lustre/lustreapi.h:     section "Present But Cannot Be Compiled"
configure: WARNING: lustre/lustreapi.h: proceeding with the compiler's result
configure: WARNING:     ## ------------------------------------------------------ ##
configure: WARNING:     ## Report this to http://www.open-mpi.org/community/help/ ##
configure: WARNING:     ## ------------------------------------------------------ ##
checking for lustre/lustreapi.h... no
configure: error: Lustre support requested but not found.  Aborting

The first and last few lines of lustreapi.h (I can send the whole file if needed):

ifndef _LUSTREAPI_H_
#define _LUSTREAPI_H_

/** \defgroup llapi llapi
 *
 * @{
 */

#include <glob.h>
#include <stdarg.h>
#include <stdint.h>
#include <time.h>
#include <unistd.h>
#include <sys/types.h>
#include <linux/lustre/lustre_user.h>

...

/** @} llapi */

#if defined(__cplusplus)
}
#endif

#endif

The only uncommented part of lustre_user.h:

#include <linux/lustre/lustre_user.h>

[edgargabriel]

Thank you for the bug report, would it be possible to upload the config.log file to see the precise error message that is generated?

[Tristan-Kosciuch]

config.log

[Tristan-Kosciuch]

lustreapi.h.gz

[edgargabriel]

Thank you, I am not entirely sure what to make of this, this looks like either a genuine bug in lustreapi.h, a misconfiguration on the platforms, or something has changed with the lustre header files that I am not aware of (Note: I do not have access to a system with this lustre version for testing)

configure:317138: gcc -c -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -mcx16 -pthread     -I/usr/local/include -I/usr/local/include -I/usr/src/lustre-client-modules-2.14.0-ddn54/lustre/include conftest.c >&5
In file included from conftest.c:779:
/usr/src/lustre-client-modules-2.14.0-ddn54/lustre/include/lustre/lustreapi.h:592:6: error: nested redefinition of 'enum llapi_json_types'
  592 | enum llapi_json_types {
      |      ^~~~~~~~~~~~~~~~
/usr/src/lustre-client-modules-2.14.0-ddn54/lustre/include/lustre/lustreapi.h:592:6: error: redeclaration of 'enum llapi_json_types'
In file included from /usr/src/lustre-client-modules-2.14.0-ddn54/lustre/include/lustre/lustreapi.h:47,
                 from conftest.c:779:
/usr/include/linux/lustre/lustre_user.h:2224:6: note: originally defined here
 2224 | enum llapi_json_types {

Would it be possible for you to inquire with DDN how to resolve this, or what the issue could be?

[Tristan-Kosciuch]

Open issues are GoogleCloudPlatform/cluster-toolkit#768 and DDNStorage/exascaler-cloud-terraform#14. Can you confirm that for my workload (which uses ompi) to work on top of a lustre filesystem, does ompi need to be built with --with-lustre=? Thank you

[edgargabriel]

You will need it if your code is doing MPI I/O on the lustre file system. It wouldn't matter if your code does some other form of file I/O, i.e. not MPI_File_read/write etc.

[Tristan-Kosciuch]

I think the error is generated while this file is being run https://github.com/open-mpi/ompi/blob/9eec56222a5c98d13790c9ee74877f1562ac27e8/config/ompi_check_lustre.m4. No response yet from DDN.

[edgargabriel]

@Tristan-Kosciuch just for explanation, this is the configure macro of Open MPI that checks for the availability of the function in the lustre header. It has worked without issues on previous lustre releases.