All tests fail on MacOS XCode-16.1 when -DOQS_KEM_ENCODERS=ON

[mouse07410]

Describe the bug
All tests fail when I add -DOQS_KEM_ENCODERS=ON to the CMake params.

To Reproduce
Steps to reproduce the behavior:

1. Clone the repo.
2. Configure with cmake -DCMAKE_BUILD_TYPE=Debug -DOQS_KEM_ENCODERS=ON -DOPENSSL_ROOT_DIR="/opt/local/libexec/openssl3" -DCMAKE_C_FLAGS="$CFLAGS -g -I/opt/local/include -L/opt/local/lib" -DCMAKE_VERBOSE_MAKEFILE:BOOL=True -S . -B _build 2>&1 | tee -a build-out.txt
3. cmake build
4. scripts/runtests.sh
5. See error

External interop tests commencing
Using Web proxy "xxxxxxxxxxxxxxx"
 Google:
  x25519_kyber768 @ oqsprovider
  X25519MLKEM768 @ oqsprovider
Test project /Users/ur20980/src/oqs-provider/_build
    Start 1: oqs_signatures
1/6 Test #1: oqs_signatures ...................Subprocess aborted***Exception:   0.01 sec
    Start 2: oqs_kems
2/6 Test #2: oqs_kems .........................Subprocess aborted***Exception:   0.01 sec
    Start 3: oqs_groups
3/6 Test #3: oqs_groups .......................Subprocess aborted***Exception:   0.01 sec
    Start 4: oqs_tlssig
4/6 Test #4: oqs_tlssig .......................Subprocess aborted***Exception:   0.01 sec
    Start 5: oqs_endecode
5/6 Test #5: oqs_endecode .....................Subprocess aborted***Exception:   0.01 sec
    Start 6: oqs_evp_pkey_params
6/6 Test #6: oqs_evp_pkey_params ..............Subprocess aborted***Exception:   0.01 sec

0% tests passed, 6 tests failed out of 6

Expected behavior

External interop tests commencing
Using Web proxy "xxxxxxxxxxxxxxx"
 Google:
  x25519_kyber768 @ oqsprovider
  X25519MLKEM768 @ oqsprovider
Test project /Users/ur20980/src/oqs-provider/_build
    Start 1: oqs_signatures
1/6 Test #1: oqs_signatures ...................   Passed    3.61 sec
    Start 2: oqs_kems
2/6 Test #2: oqs_kems .........................   Passed    0.25 sec
    Start 3: oqs_groups
3/6 Test #3: oqs_groups .......................   Passed    0.37 sec
    Start 4: oqs_tlssig
4/6 Test #4: oqs_tlssig .......................   Passed  120.36 sec
    Start 5: oqs_endecode
5/6 Test #5: oqs_endecode .....................   Passed    9.03 sec
    Start 6: oqs_evp_pkey_params
6/6 Test #6: oqs_evp_pkey_params ..............   Passed    0.70 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 136.54 sec

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: MacOS Sonoma 14.7.1
• OpenSSL version 3.4.0 22 Oct 2024 (Library: OpenSSL 3.4.0 22 Oct 2024)
• oqsprovider 0.7.1-dev

Additional context

Crashes are:

Process:               oqs_test_endecode [26776]
Path:                  /Users/USER/*/oqs_test_endecode
Identifier:            oqs_test_endecode
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        ctest [26771]
Responsible:           Terminal [4859]
User ID:               0

Date/Time:             2024-12-10 10:52:17.4068 -0500
OS Version:            macOS 14.7.1 (23H222)
Report Version:        12
Bridge OS Version:     9.1 (22P1072)
Anonymous UUID:        BD844EB9-9C6F-867E-78EB-1ACDA55970A0


Time Awake Since Boot: 390000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 6 Abort trap: 6
Terminating Process:   oqs_test_endecode [26776]

Application Specific Information:
abort() called

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff812ee7d96 __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff812f20ebd pthread_kill + 262
2   libsystem_c.dylib             	    0x7ff812e46a79 abort + 126
3   libcrypto.3.dylib             	       0x1072ab81a OPENSSL_die + 26 (cryptlib.c:260)
4   oqs_test_endecode             	       0x10655cafb load_oqs_provider + 123 (test_common.c:55)
5   oqs_test_endecode             	       0x10655c06d main + 77 (oqs_test_endecode.c:255)
6   dyld                          	    0x7ff812b95345 start + 1909

and

Process:               oqs_test_kems [26773]
Path:                  /Users/USER/*/oqs_test_kems
Identifier:            oqs_test_kems
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        ctest [26771]
Responsible:           Terminal [4859]
User ID:               0

Date/Time:             2024-12-10 10:52:17.3092 -0500
OS Version:            macOS 14.7.1 (23H222)
Report Version:        12
Bridge OS Version:     9.1 (22P1072)
Anonymous UUID:        BD844EB9-9C6F-867E-78EB-1ACDA55970A0


Time Awake Since Boot: 390000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 6 Abort trap: 6
Terminating Process:   oqs_test_kems [26773]

Application Specific Information:
abort() called


Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff812ee7d96 __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff812f20ebd pthread_kill + 262
2   libsystem_c.dylib             	    0x7ff812e46a79 abort + 126
3   libcrypto.3.dylib             	       0x1078ab81a OPENSSL_die + 26 (cryptlib.c:260)
4   oqs_test_kems                 	       0x106b58cfb load_oqs_provider + 123 (test_common.c:55)
5   oqs_test_kems                 	       0x106b583f4 main + 84 (oqs_test_kems.c:89)
6   dyld                          	    0x7ff812b95345 start + 1909

Without -DOQS_KEM_ENCODERS=ON all the tests execute and complete successfully.

Update

Switching from Xcode-16.1 Clang to Macports-installed Clang seemed to provide a workaround:

Test project /Users/ur20980/src/oqs-provider/_build
    Start 1: oqs_signatures
1/6 Test #1: oqs_signatures ...................   Passed    3.71 sec
    Start 2: oqs_kems
2/6 Test #2: oqs_kems .........................   Passed    0.31 sec
    Start 3: oqs_groups
3/6 Test #3: oqs_groups .......................   Passed    0.44 sec
    Start 4: oqs_tlssig
4/6 Test #4: oqs_tlssig .......................   Passed  218.33 sec
    Start 5: oqs_endecode
5/6 Test #5: oqs_endecode .....................   Passed   10.22 sec
    Start 6: oqs_evp_pkey_params
6/6 Test #6: oqs_evp_pkey_params ..............   Passed    1.25 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 235.79 sec

[baentsch]

Hmm, we have a test for OQS_KEM_ENCODERS=ON in CI and it passes.

Switching from Xcode-16.1 Clang to Macports-installed Clang seemed to provide a workaround:

That seems to imply the choice of Clang origin on MacOS triggers this problem and you have found a sensible workaround. IMO that does not warrant (make appear prudent?) a change to oqsprovider.

Reminder: oqsprovider has no "guaranteed supported platforms" (no community activity on #293), so this does not look like a bug in my eyes and I'll move this to discussion for others to find this workaround as/if needed (unless someone is willing to work on/for supporting MacOS-origin Xcode-16.1 Clang and (documenting that in) #293 (?)).

[mouse07410]

OK, fair enough. Let's see if Xcode-16.2 or such would remedy the problem. In the meanwhile, the workaround works.