spdk target exits when running on system without AVX 512

[seroyer]

$ docker run -it --rm --privileged -v /dev/hugepages:/dev/hugepages -v /dev/shm:/dev/shm -v /proc:/proc ghcr.io/opiproject/opi-spdk:main bash
[root@3c9db4b50106 ~]# sync; echo 1 > /proc/sys/vm/drop_caches  && \
            echo 1024 > /proc/sys/vm/nr_hugepages && \
            grep "" /sys/kernel/mm/hugepages/hugepages-*/nr_hugepages
/sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages:0
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages:1024
[root@3c9db4b50106 ~]# /usr/local/bin/spdk_tgt -m 0x1 -s 512 --no-pci
Illegal instruction (core dumped)

The VMs which we are using to build SPDK and push to GHCR have AVX512. So if you try to run the container on a system that does not have AVX 512, it will fail.

[mestery]

See here for how I proposed fixing this for p4-dpdk.

[glimchb]

for SPDK you can try

diff --git a/storage/Dockerfile b/storage/Dockerfile
index df7f06e..a836763 100644
--- a/storage/Dockerfile
+++ b/storage/Dockerfile
@@ -1,7 +1,7 @@
 FROM fedora:36 as build

 ARG TAG=v22.05
-ARG ARCH=native
+ARG ARCH=x86_64

 WORKDIR /root
 RUN dnf install -y git rpm-build diffutils procps-ng && dnf clean all
(END)

because

mk/spdk.common.mk-73-ifeq ($(TARGET_MACHINE),x86_64)
mk/spdk.common.mk:74:# Don't use AVX-512 instructions in SPDK code - it breaks Valgrind for
mk/spdk.common.mk-75-# some cases where compiler decides to hyper-optimize a relatively
mk/spdk.common.mk:76:# simple operation (like int-to-float converstion) using AVX-512
mk/spdk.common.mk:77:COMMON_CFLAGS += -mno-avx512f
mk/spdk.common.mk-78-endif

[mestery]

So folks are aware, the way to test this is as follows:

• Push a PR with the fix.
• The CI should build and push a container to GHCR
• Locally modify the Dockerfile to use the PR image
• Verify the fix

If all that works, we can merge this.

[mestery]

for SPDK you can try

diff --git a/storage/Dockerfile b/storage/Dockerfile
index df7f06e..a836763 100644
--- a/storage/Dockerfile
+++ b/storage/Dockerfile
@@ -1,7 +1,7 @@
 FROM fedora:36 as build

 ARG TAG=v22.05
-ARG ARCH=native
+ARG ARCH=x86_64

 WORKDIR /root
 RUN dnf install -y git rpm-build diffutils procps-ng && dnf clean all
(END)

because

mk/spdk.common.mk-73-ifeq ($(TARGET_MACHINE),x86_64)
mk/spdk.common.mk:74:# Don't use AVX-512 instructions in SPDK code - it breaks Valgrind for
mk/spdk.common.mk-75-# some cases where compiler decides to hyper-optimize a relatively
mk/spdk.common.mk:76:# simple operation (like int-to-float converstion) using AVX-512
mk/spdk.common.mk:77:COMMON_CFLAGS += -mno-avx512f
mk/spdk.common.mk-78-endif

Trying this now and following my instructions on verifying before we merge as above.

[mestery]

Verified the pushed container image and the tests pass locally on my laptop using a Virtualbox VM.