current master fails to configure - libprov dir is empty #359

mouse07410 · 2021-10-14T05:09:19Z

MacOS Big Sur 11.6, Xcode-13.0 (both factors irrelevant, IMHO).

+ LDFLAGS=
+ OPENSSL_DIR=/Users/uri/openssl-3
+ OPENSSL_ENGINES_DIR=/Users/uri/openssl-3/lib/engines-3
+ THREE=-3-
+ '[' -z clang ']'
+ '[' -z '' ']'
+ export OPENSSL_ROOT_DIR=/Users/uri/openssl-3
+ OPENSSL_ROOT_DIR=/Users/uri/openssl-3
+ '[' -z ']'
+ CMAKE_BUILD_TYPE=Release
+ OPENSSL_INCLUDE_DIR=/Users/uri/openssl-3/include
+ OPENSSL_CRYPTO_LIBRARY=/Users/uri/openssl-3/lib/libcrypto.dylib
+ OPENSSL_SSL_LIBRARY=/Users/uri/openssl-3/lib/libssl.dylib
+ PKG_CONFIG_PATH=/Users/uri/openssl-3/lib/pkgconfig:/opt/local/lib/pkgconfig:/opt/local/sha
re/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
+ OPENSSL_CFLAGS='-march=native -std=gnu17'
+ OPENSSL_LIB_DIR=/Users/uri/openssl-3/lib
+ OPENSSL_CONF=/Users/uri/openssl-3/etc/openssl.cnf
+ rm -rf build
+ mkdir -p build
+ cd build
+ cmake .. -DCMAKE_C_COMPILER=clang -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE:BOOL
=ON -DOPENSSL_ROOT_DIR=/Users/uri/openssl-3 -DOPENSSL_ENGINES_DIR=/Users/uri/openssl-3/lib/e
ngines-3
-- The C compiler identification is AppleClang 13.0.0.13000029
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/local/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found OpenSSL: /Users/uri/openssl-3/lib/libcrypto.dylib (found suitable version "3.1.0", minimum required is "3.0")  
-- Found OpenSSL ...
CMAKE_BUILD_TYPE = Release
-- OpenSSL version values were not properly defined (OpenSSL-3 issue), setting to 3.0
-- Found OpenSSL includes:   /Users/uri/openssl-3/include
-- Found OpenSSL crypto lib: /Users/uri/openssl-3/lib/libcrypto.dylib
-- Found OpenSSL application: 
-- Found OpenSSL application: /Users/uri/openssl-3/bin/openssl
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - not found
-- LITTLE_ENDIAN
-- Performing Test ADDCARRY_U64
-- Performing Test ADDCARRY_U64 - Success
-- Performing Test RELAXED_ALIGNMENT
-- Performing Test RELAXED_ALIGNMENT - Success
-- No Test2::V0 perl module (engine and provider tests skipped)
CMake Error at CMakeLists.txt:376 (add_subdirectory):
  The source directory

    /Users/uri/src/grasshopper-engine/libprov

  does not contain a CMakeLists.txt file.


INSTALL TARGETS - target lib_gost_engine has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
-- Configuring incomplete, errors occurred!

See also "/Users/uri/src/grasshopper-engine/build/CMakeFiles/CMakeOutput.log".
See also "/Users/uri/src/grasshopper-engine/build/CMakeFiles/CMakeError.log".

The text was updated successfully, but these errors were encountered:

beldmit · 2021-10-14T05:25:55Z

You should perform
git submodule update --init to fetch the libprov submodule

mouse07410 · 2021-10-14T05:27:28Z

Update

After pulling libprov submodule, master configures and builds - but fails keyexpimp test:

.  .  .  .  .
test 12
      Start 12: keyexpimp

12: Test command: /Users/uri/src/grasshopper-engine/build/bin/test_keyexpimp
12: Test timeout computed to be: 10000000
12/13 Test #12: keyexpimp ........................***Exception: SegFault  0.01 sec
.  .  .  .  .

Crash report:

Process:               test_keyexpimp [54239]
Path:                  /Users/USER/*/test_keyexpimp
Identifier:            test_keyexpimp
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        ctest [54209]
Responsible:           Terminal [583]
User ID:               501

Date/Time:             2021-10-14 01:11:28.698 -0400
OS Version:            macOS 11.6 (20G165)
Report Version:        12
Anonymous UUID:        B3544DF6-AE65-3AD0-0E52-DD6C16797F5A

Sleep/Wake UUID:       73CBEBA0-C23E-4C74-B7C6-9C1A282421FA

Time Awake Since Boot: 1000000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x00000000000004a6
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [54239]

VM Regions Near 0x4a6:
--> 
    __TEXT                      1004a6000-1004aa000    [   16K] r-x/r-x SM=COW  /Users/*

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libcrypto.3.dylib             	0x0000000100640b72 ossl_lh_strcasehash + 18 (lhash.c:370)
1   libcrypto.3.dylib             	0x000000010066507d obj_name_hash + 61 (o_names.c:157)
2   libcrypto.3.dylib             	0x0000000100640987 getrn + 8 (lhash.c:311) [inlined]
3   libcrypto.3.dylib             	0x0000000100640987 OPENSSL_LH_retrieve + 39 (lhash.c:173)
4   libcrypto.3.dylib             	0x0000000100664b16 lh_OBJ_NAME_retrieve + 5 (obj_local.h:12) [inlined]
5   libcrypto.3.dylib             	0x0000000100664b16 OBJ_NAME_get + 118 (o_names.c:183)
6   libcrypto.3.dylib             	0x000000010062a60c evp_get_cipherbyname_ex + 29 (names.c:85) [inlined]
7   libcrypto.3.dylib             	0x000000010062a60c EVP_get_cipherbyname + 44 (names.c:72)
8   test_keyexpimp                	0x00000001004a82c8 omac_imit_ctrl + 616
9   test_keyexpimp                	0x00000001004a8865 gost_kexp15 + 261
10  test_keyexpimp                	0x00000001004a7b57 main + 423
11  libdyld.dylib                 	0x00007fff20350f3d start + 1

When compiling engine with "-g", crash does not occur. It seems a compiler (Clang) bug, but a workaround would be nice, at least until Apple fixes it. I suspect it's related to -msse4.1, in case it matters.

beldmit · 2021-10-14T17:19:34Z

Don't know how to deal with it... Do you have something like valgrind on mac?

mouse07410 · 2021-10-14T17:28:09Z

Do you have something like valgrind on mac?

Alas, no. Might be able to use sanitizers, but...

On a different MacOS machine (the most powerful of my Mac herd ;), I'm getting Provider test failing on the master (and keyexpimp working). Current (as of today) OpenSSL 3.0 dev (aka, master).

15/15 Testing: provider
15/15 Test: provider
Command: "/opt/local/bin/prove" "--merge" "-PWrapOpenSSL" "/Users/ur20980/src/grasshopper-engine/test" "::" "provider"
Directory: /Users/ur20980/src/grasshopper-engine/build
"provider" start time: Oct 14 12:16 EDT
Output:
----------------------------------------------------------
Added /Users/ur20980/openssl-3/lib to:
  LD_LIBRARY_PATH, DYLD_LIBRARY_PATH, LIBPATH
Added /Users/ur20980/openssl-3/bin to:
  PATH
LD_LIBRARY_PATH=/Users/ur20980/openssl-3/lib
DYLD_LIBRARY_PATH=/Users/ur20980/openssl-3/lib
LIBPATH=/Users/ur20980/openssl-3/lib
PATH=/Users/ur20980/openssl-3/bin:/opt/local/bin:/opt/local/sbin:/opt/local/libexec/gnubin:/opt/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/Library/TeX/texbin:/usr/local/go/bin:/usr/local/MacGPG2/bin:/Library/OpenSC/bin:/usr/local/share/dotnet:/opt/X11/bin:~/.dotnet/tools:/Library/Apple/usr/bin:/Applications/Wireshark.app/Contents/MacOS:/Users/ur20980/.cargo/bin:/Users/ur20980/.cabal/bin:/Users/ur20980/.ghcup/bin:/Users/ur20980/.local/bin:/Users/ur20980/bin:/opt/local/libexec/qt5/bin:/Library/Java/NSS/bin:/Users/ur20980/go/bin:/opt/jruby/bin:/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin:/Users/ur20980/Library/Python/3.8/bin:/opt/GNAT/2019/bin:/Users/ur20980/src/OpenDDS/ACE_TAO/ACE/bin:/Users/ur20980/src/OpenDDS/bin
/Users/ur20980/src/grasshopper-engine/test/00-engine.t .... skipped: This test is only suitable for the engine
/Users/ur20980/src/grasshopper-engine/test/00-provider.t .. 
# Seeded srand with seed '20211014' from local date.
1..1
ok 1 - load provider without any config
ok
/Users/ur20980/src/grasshopper-engine/test/01-digest.t .... 
# Seeded srand with seed '20211014' from local date.
1..16
# Running tests for test type conf
ok 1 - GOST R 34.11-2012 256bit example 1 from standard
ok 2 - GOST R 34.11-2012 512bit example 1 from standard
ok 3 - GOST R 34.11-2012 256bit example 2 from standard
ok 4 - GOST R 34.11-2012 512bit example 2 from standard
ok 5 - GOST R 34.11-94 1K ascii
ok 6 - GOST R 34.11-2012 256bit 1K ascii
ok 7 - GOST R 34.11-2012 512bit 1K ascii
ok 8 - GOST R 34.11-94 1K binary
ok 9 - GOST R 34.11-2012 256bit 1K binary
ok 10 - GOST R 34.11-2012 512bit 1K binary
ok 11 - GOST R 34.11-94 539 bytes
ok 12 - GOST R 34.11-2012 256bit 539 bytes
ok 13 - GOST R 34.11-2012 512bit 539 bytes
ok 14 - GOST R 34.11-94 128K
ok 15 - GOST R 34.11-2012 256bit 128K
ok 16 - GOST R 34.11-2012 512bit 128K
ok
/Users/ur20980/src/grasshopper-engine/test/02-mac.t ....... 
# Seeded srand with seed '20211014' from local date.
1..19
# @ARGV = (provider)
# Running tests for test type conf
ok 1 - GOST MAC - default size
ok 2 - GOST MAC - size 1 bytes
ok 3 - GOST MAC - size 2 bytes
ok 4 - GOST MAC - size 3 bytes
ok 5 - GOST MAC - size 4 bytes
not ok 6 - GOST MAC - size 5 bytes

# Failed test 'GOST MAC - size 5 bytes'
# at /Users/ur20980/src/grasshopper-engine/test/02-mac.t line 174.
# +--------------+----+--------------+
# | GOT          | OP | CHECK        |
# +--------------+----+--------------+
# | 2EE8D13D35\n | eq | 2EE8D13DFF\n |
# +--------------+----+--------------+
# Command was: openssl mac  -macopt key:0123456789abcdef0123456789abcdef -macopt size:5 -in testdata.dat gost-mac
not ok 7 - GOST MAC - size 6 bytes

# Failed test 'GOST MAC - size 6 bytes'
# at /Users/ur20980/src/grasshopper-engine/test/02-mac.t line 174.
# +----------------+----+----------------+
# | GOT            | OP | CHECK          |
# +----------------+----+----------------+
# | 2EE8D13D3536\n | eq | 2EE8D13DFF7F\n |
# +----------------+----+----------------+
# Command was: openssl mac  -macopt key:0123456789abcdef0123456789abcdef -macopt size:6 -in testdata.dat gost-mac
not ok 8 - GOST MAC - size 7 bytes

# Failed test 'GOST MAC - size 7 bytes'
# at /Users/ur20980/src/grasshopper-engine/test/02-mac.t line 174.
# +------------------+----+------------------+
# | GOT              | OP | CHECK            |
# +------------------+----+------------------+
# | 2EE8D13D353637\n | eq | 2EE8D13DFF7F03\n |
# +------------------+----+------------------+
# Command was: openssl mac  -macopt key:0123456789abcdef0123456789abcdef -macopt size:7 -in testdata.dat gost-mac
not ok 9 - GOST MAC - size 8 bytes

# Failed test 'GOST MAC - size 8 bytes'
# at /Users/ur20980/src/grasshopper-engine/test/02-mac.t line 174.
# +--------------------+----+--------------------+
# | GOT                | OP | CHECK              |
# +--------------------+----+--------------------+
# | 2EE8D13D35363730\n | eq | 2EE8D13DFF7F037D\n |
# +--------------------+----+--------------------+
# Command was: openssl mac  -macopt key:0123456789abcdef0123456789abcdef -macopt size:8 -in testdata.dat gost-mac
ok 10 - GOST MAC - big data
ok 11 - GOST MAC parameters 2012 - default size
ok 12 - GOST MAC parameters 2012 - size 1 bytes
ok 13 - GOST MAC parameters 2012 - size 2 bytes
ok 14 - GOST MAC parameters 2012 - size 3 bytes
ok 15 - GOST MAC parameters 2012 - size 4 bytes
not ok 16 - GOST MAC parameters 2012 - size 5 bytes

# Failed test 'GOST MAC parameters 2012 - size 5 bytes'
# at /Users/ur20980/src/grasshopper-engine/test/02-mac.t line 204.
# +--------------+----+--------------+
# | GOT          | OP | CHECK        |
# +--------------+----+--------------+
# | BE4453EC35\n | eq | BE4453EC1E\n |
# +--------------+----+--------------+
# Command was: openssl mac  -macopt key:0123456789abcdef0123456789abcdef -macopt size:5 -in testdata.dat gost-mac-12
not ok 17 - GOST MAC parameters 2012 - size 6 bytes

# Failed test 'GOST MAC parameters 2012 - size 6 bytes'
# at /Users/ur20980/src/grasshopper-engine/test/02-mac.t line 204.
# +----------------+----+----------------+
# | GOT            | OP | CHECK          |
# +----------------+----+----------------+
# | BE4453EC3536\n | eq | BE4453EC1EC3\n |
# +----------------+----+----------------+
# Command was: openssl mac  -macopt key:0123456789abcdef0123456789abcdef -macopt size:6 -in testdata.dat gost-mac-12
not ok 18 - GOST MAC parameters 2012 - size 7 bytes

# Failed test 'GOST MAC parameters 2012 - size 7 bytes'
# at /Users/ur20980/src/grasshopper-engine/test/02-mac.t line 204.
# +------------------+----+------------------+
# | GOT              | OP | CHECK            |
# +------------------+----+------------------+
# | BE4453EC353637\n | eq | BE4453EC1EC327\n |
# +------------------+----+------------------+
# Command was: openssl mac  -macopt key:0123456789abcdef0123456789abcdef -macopt size:7 -in testdata.dat gost-mac-12
not ok 19 - GOST MAC parameters 2012 - size 8 bytes

# Failed test 'GOST MAC parameters 2012 - size 8 bytes'
# at /Users/ur20980/src/grasshopper-engine/test/02-mac.t line 204.
# +--------------------+----+--------------------+
# | GOT                | OP | CHECK              |
# +--------------------+----+--------------------+
# | BE4453EC35363730\n | eq | BE4453EC1EC327BE\n |
# +--------------------+----+--------------------+
# Command was: openssl mac  -macopt key:0123456789abcdef0123456789abcdef -macopt size:8 -in testdata.dat gost-mac-12
Dubious, test returned 8 (wstat 2048, 0x800)
Failed 8/19 subtests 
/Users/ur20980/src/grasshopper-engine/test/03-encrypt.t ... 
# Seeded srand with seed '20211014' from local date.
1..48
# Running tests for test type conf
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 1 - CFB short text, paramset A - Trying to encrypt
ok 2 - CFB short text, paramset A - Checking that it encrypted correctly
ok 3 - CFB short text, paramset A - Trying to decrypt
ok 4 - CFB short text, paramset A - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 5 - CFB short text, paramset B - Trying to encrypt
ok 6 - CFB short text, paramset B - Checking that it encrypted correctly
ok 7 - CFB short text, paramset B - Trying to decrypt
ok 8 - CFB short text, paramset B - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 9 - CFB short text, paramset C - Trying to encrypt
ok 10 - CFB short text, paramset C - Checking that it encrypted correctly
ok 11 - CFB short text, paramset C - Trying to decrypt
ok 12 - CFB short text, paramset C - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 13 - CFB short text, paramset D - Trying to encrypt
ok 14 - CFB short text, paramset D - Checking that it encrypted correctly
ok 15 - CFB short text, paramset D - Trying to decrypt
ok 16 - CFB short text, paramset D - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 17 - CNT short text - Trying to encrypt
ok 18 - CNT short text - Checking that it encrypted correctly
ok 19 - CNT short text - Trying to decrypt
ok 20 - CNT short text - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 21 - CNT short text, paramset param doesnt affect cnt - Trying to encrypt
ok 22 - CNT short text, paramset param doesnt affect cnt - Checking that it encrypted correctly
ok 23 - CNT short text, paramset param doesnt affect cnt - Trying to decrypt
ok 24 - CNT short text, paramset param doesnt affect cnt - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 25 - CNT-12 short text - Trying to encrypt
ok 26 - CNT-12 short text - Checking that it encrypted correctly
ok 27 - CNT-12 short text - Trying to decrypt
ok 28 - CNT-12 short text - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 29 - CNT-12 short text, paramset param doesnt affect cnt - Trying to encrypt
ok 30 - CNT-12 short text, paramset param doesnt affect cnt - Checking that it encrypted correctly
ok 31 - CNT-12 short text, paramset param doesnt affect cnt - Trying to decrypt
ok 32 - CNT-12 short text, paramset param doesnt affect cnt - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 33 - CBC short text, paramset A - Trying to encrypt
ok 34 - CBC short text, paramset A - Checking that it encrypted correctly
ok 35 - CBC short text, paramset A - Trying to decrypt
ok 36 - CBC short text, paramset A - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 37 - CBC short text, paramset B - Trying to encrypt
ok 38 - CBC short text, paramset B - Checking that it encrypted correctly
ok 39 - CBC short text, paramset B - Trying to decrypt
ok 40 - CBC short text, paramset B - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 41 - CBC short text, paramset C - Trying to encrypt
ok 42 - CBC short text, paramset C - Checking that it encrypted correctly
ok 43 - CBC short text, paramset C - Trying to decrypt
ok 44 - CBC short text, paramset C - Checking that it decrypted correctly
Use of uninitialized value $test_type in concatenation (.) or string at /Users/ur20980/src/grasshopper-engine/test/03-encrypt.t line 81.
ok 45 - CBC short text, paramset D - Trying to encrypt
ok 46 - CBC short text, paramset D - Checking that it encrypted correctly
ok 47 - CBC short text, paramset D - Trying to decrypt
ok 48 - CBC short text, paramset D - Checking that it decrypted correctly
ok
/Users/ur20980/src/grasshopper-engine/test/04-pkey.t ...... skipped: TODO: add pkey support in provider

Test Summary Report
-------------------
/Users/ur20980/src/grasshopper-engine/test/02-mac.t     (Wstat: 2048 Tests: 19 Failed: 8)
  Failed tests:  6-9, 16-19
  Non-zero exit status: 8
Files=6, Tests=84,  1 wallclock secs ( 0.03 usr  0.02 sys +  0.68 cusr  0.36 csys =  1.09 CPU)
Result: FAIL
<end of output>
Test time =   1.19 sec
----------------------------------------------------------
Test Failed.
"provider" end time: Oct 14 12:16 EDT
"provider" time elapsed: 00:00:01

beldmit · 2021-10-14T20:54:55Z

@levitte, looks like a Mac-specific(?) problem with XOF...

levitte · 2021-10-16T01:17:37Z

Noted. I've seen something similar (not XOF related, though) in another provider I wrote. Maybe the reasons are similar... I'll have a look in a couple of days.

mouse07410 · 2021-10-17T00:33:57Z

I wonder if it's in any way related to the cause of this: randombit/botan#2802 (compiler bug)?

levitte · 2021-10-17T04:33:03Z

It's a bit interesting, though, that the Github Actions don't fail. I wonder why that is...

Why I see on the provider test failure above is that the size macopt doesn't seem to go through all the way. The implementation itself defaults at 4:

engine/gost_crypt.c

Line 1213 in df3ead2

c->dgst_size = 4;

... and my current working hypothesis is that somehow, the attempt to change that doesn't quite work:

engine/gost_prov_mac.c

Line 147 in df3ead2

EVP_MD_CTX_ctrl(gctx->dctx, EVP_MD_CTRL_XOF_LEN, gctx->mac_size, NULL);

I wonder if this might be a type coersion error in the compiler. gctx->mac_size is a size_t, while EVP_MD_CTX_ctrl() takes an int in that position... so, guessing wildly here, could it be that arg inside EVP_MD_CTX_ctrl() has become zero? That would leave the XOF size unchanged (i.e. still 4), and get the exact (wrong) result seen above...

Does the result change with this simple patch?

diff --git a/gost_prov_mac.c b/gost_prov_mac.c
index ab04a9c..0948d37 100644
--- a/gost_prov_mac.c
+++ b/gost_prov_mac.c
@@ -144,7 +144,7 @@ static int mac_final(void *mctx, unsigned char *out, size_t *outl,
     if (out != NULL) {
         /* We ignore the error for GOST MDs that don't support setting
            the size */
-        EVP_MD_CTX_ctrl(gctx->dctx, EVP_MD_CTRL_XOF_LEN, gctx->mac_size, NULL);
+        EVP_MD_CTX_ctrl(gctx->dctx, EVP_MD_CTRL_XOF_LEN, (int)gctx->mac_size, NULL);
         ret = EVP_DigestFinal_ex(gctx->dctx, out, &tmpoutl);
     }
     if (outl != NULL)

beldmit · 2021-10-17T07:34:15Z

I wonder if it's in any way related to the cause of this: randombit/botan#2802 (compiler bug)?

I believe the test_keyexpimp failure may be related, but I thinkr @levitte's hypothesis is more relevant

mouse07410 · 2021-10-17T12:57:35Z

It's a bit interesting, though, that the Github Actions don't fail. I wonder why that is...

Because they don't use Clang with a CPU that has SSE4.1 enabled or detected? Or because their Clang is not v12?

mouse07410 · 2021-10-17T13:59:03Z

Does the result change with this simple patch?

I'm afraid not:

Running tests...
/opt/local/bin/ctest --force-new-ctest-process 
Test project /Users/ur20980/src/grasshopper-engine/build
      Start  1: digest-with-engine
 1/15 Test  #1: digest-with-engine ...............   Passed    0.07 sec
      Start  2: digest-with-provider
 2/15 Test  #2: digest-with-provider .............   Passed    0.06 sec
      Start  3: ciphers-with-engine
 3/15 Test  #3: ciphers-with-engine ..............   Passed    0.04 sec
      Start  4: ciphers-with-provider
 4/15 Test  #4: ciphers-with-provider ............   Passed    0.04 sec
      Start  5: curves
 5/15 Test  #5: curves ...........................   Passed    0.04 sec
      Start  6: parameters-with-engine
 6/15 Test  #6: parameters-with-engine ...........   Passed    0.03 sec
      Start  7: derive-with-engine
 7/15 Test  #7: derive-with-engine ...............   Passed    0.06 sec
      Start  8: sign/verify-with-engine
 8/15 Test  #8: sign/verify-with-engine ..........   Passed    0.12 sec
      Start  9: TLS-with-engine
 9/15 Test  #9: TLS-with-engine ..................   Passed    0.71 sec
      Start 10: context-with-engine
10/15 Test #10: context-with-engine ..............   Passed    0.02 sec
      Start 11: context-with-provider
11/15 Test #11: context-with-provider ............   Passed    0.01 sec
      Start 12: keyexpimp
12/15 Test #12: keyexpimp ........................   Passed    0.02 sec
      Start 13: gost89
13/15 Test #13: gost89 ...........................   Passed    0.02 sec
      Start 14: engine
14/15 Test #14: engine ...........................   Passed    3.56 sec
      Start 15: provider
15/15 Test #15: provider .........................***Failed    1.64 sec

93% tests passed, 1 tests failed out of 15

Total Test time (real) =   6.46 sec

The following tests FAILED:
	 15 - provider (Failed)
Errors while running CTest
Output from these tests are in: /Users/ur20980/src/grasshopper-engine/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

And the complete log (including configuration and build):
ossl3-build.txt

mouse07410 · 2021-10-25T14:07:31Z

Ping...?

beldmit · 2021-10-26T16:26:50Z

Will try to play with the new clang on Friday

beldmit · 2021-10-28T12:05:05Z

Using clang 12.0.1, I get segfault on digest-with-provider. Will try to investigate.

beldmit · 2021-10-28T12:17:10Z

Test output:

Test md_gost12_512: M1 from RFC 6986 (10.1.1) and GOST R 34.11-2012 (А.1.1): [MD 64] success
Test md_gost12_256: M1 from RFC 6986 (10.1.2) and GOST R 34.11-2012 (А.1.2): [MD 32] success
Test md_gost12_512: M2 from RFC 6986 (10.2.1) and GOST R 34.11-2012 (А.2.1): [MD 0] success
Test md_gost12_256: M2 from RFC 6986 (10.2.2) and GOST R 34.11-2012 (А.2.2): [MD 0] success
806BEB56E77F0000:error:0300009B:digital envelope routines:evp_mac_final:buffer too small:crypto/evp/mac_lib.c:156:
/home/dbelyavs/upstream/engine/test_digest.c:666: OpenSSL internal error: EVP_MAC_finalXOF(ctx, out, outsize)

mouse07410 · 2021-11-10T01:50:44Z

test-3-out.txt
make-3-out.txt
cmake-3-out.txt
test-3-long-out.txt

beldmit · 2021-11-14T10:48:29Z

I have just patched the test that was failing on linux. Could you please retry?

mouse07410 · 2021-11-14T16:11:01Z

digest-with-provider test seems fixed.
provider test reliably fails for OpenSSL-3.x.

Here's with the OpenSSL-3.0.0 (released):
cmake-3-out.txt
make-3-out.txt
test-3-out.txt
test-3-long-out.txt

Here's with the OpenSSL-3.1.dev (master):
test-3m-long-out.txt
test-3m-out.txt
make-3m-out.txt
cmake-3m-out.txt

And I still think that renaming "grashopper" to "kuznyechik", especially with that spelling, was a bad idea. As they say, "ugly as a mule". ;-)

beldmit · 2021-11-14T18:10:16Z

I think I found the culprit:

engine/gost_prov_mac.c

Line 148 in 8efda32

ret = EVP_DigestFinal_ex(gctx->dctx, out, &tmpoutl);

It should be EVP_DigestFinalXOF, but naïve replacement does not help.

beldmit · 2021-11-14T18:43:01Z

Could you please the corresponding fix?
I still can't reproduce it but it should fix the issue.

index ab04a9c..1bad120 100644
--- a/gost_prov_mac.c
+++ b/gost_prov_mac.c
@@ -145,7 +145,7 @@ static int mac_final(void *mctx, unsigned char *out, size_t *outl,
         /* We ignore the error for GOST MDs that don't support setting
            the size */
         EVP_MD_CTX_ctrl(gctx->dctx, EVP_MD_CTRL_XOF_LEN, gctx->mac_size, NULL);
-        ret = EVP_DigestFinal_ex(gctx->dctx, out, &tmpoutl);
+        ret = EVP_DigestFinalXOF(gctx->dctx, out, gctx->mac_size);
     }
     if (outl != NULL)
         *outl = (size_t)gctx->mac_size;

mouse07410 · 2021-11-14T23:45:01Z

Funny. This change made things worse - now two tests fail: context-with-provider and provider.

ossl3m-build.txt
ossl3-build.txt

Crash report doesn't help much:

Process:               test_context [9918]
Path:                  /Users/USER/*/test_context
Identifier:            test_context
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        ctest [9889]
Responsible:           Terminal [813]
User ID:               501

Date/Time:             2021-11-14 18:42:48.627 -0500
OS Version:            macOS 11.6.1 (20G224)
Report Version:        12
Bridge OS Version:     6.0 (19P548)
Anonymous UUID:        BD844EB9-9C6F-867E-78EB-1ACDA55970A0


Time Awake Since Boot: 380000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff204e592e __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff205145bd pthread_kill + 263
2   libsystem_c.dylib             	0x00007fff20469406 abort + 125
3   libcrypto.3.dylib             	0x00000001077bdc3f OPENSSL_die + 26
4   test_context                  	0x000000010768d226 main + 1670
5   libdyld.dylib                 	0x00007fff2052ff3d start + 1

beldmit · 2021-11-15T08:10:41Z

Do I correctly assume that you've updated the code and applied the patch afterwards?

mouse07410 · 2021-11-15T08:53:49Z

Do I correctly assume that you've updated the code and applied the patch afterwards?

Yes. But I don't see any updates...? Latest commit from you that I pulled is 8efda32. And on the libprov submodule: 8a126e09547.

beldmit · 2021-11-15T09:02:22Z

Yes, it's a relevant state +patch from #359 (comment)

mouse07410 · 2021-11-15T09:18:52Z

Yes, that's the current status of the code on my machines. Plus, same error with both OpenSSL_3.0.0 and 3.1.dev.

beldmit mentioned this issue Oct 28, 2021

XOF interface is unsuitable for truncated MACs openssl/openssl#16934

Closed

mouse07410 added a commit to mouse07410/engine that referenced this issue Nov 19, 2021

Apply patch from gost-engine#359

efbabe4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

current master fails to configure - libprov dir is empty #359

current master fails to configure - libprov dir is empty #359

mouse07410 commented Oct 14, 2021

beldmit commented Oct 14, 2021

mouse07410 commented Oct 14, 2021

beldmit commented Oct 14, 2021

mouse07410 commented Oct 14, 2021

beldmit commented Oct 14, 2021

levitte commented Oct 16, 2021

mouse07410 commented Oct 17, 2021

levitte commented Oct 17, 2021

beldmit commented Oct 17, 2021

mouse07410 commented Oct 17, 2021 •

edited

Loading

mouse07410 commented Oct 17, 2021

mouse07410 commented Oct 25, 2021

beldmit commented Oct 26, 2021

beldmit commented Oct 28, 2021

beldmit commented Oct 28, 2021

mouse07410 commented Nov 10, 2021

beldmit commented Nov 14, 2021

mouse07410 commented Nov 14, 2021

beldmit commented Nov 14, 2021

beldmit commented Nov 14, 2021

mouse07410 commented Nov 14, 2021 •

edited

Loading

beldmit commented Nov 15, 2021

mouse07410 commented Nov 15, 2021 •

edited

Loading

beldmit commented Nov 15, 2021

mouse07410 commented Nov 15, 2021

current master fails to configure - libprov dir is empty #359

current master fails to configure - libprov dir is empty #359

Comments

mouse07410 commented Oct 14, 2021

beldmit commented Oct 14, 2021

mouse07410 commented Oct 14, 2021

Update

beldmit commented Oct 14, 2021

mouse07410 commented Oct 14, 2021

beldmit commented Oct 14, 2021

levitte commented Oct 16, 2021

mouse07410 commented Oct 17, 2021

levitte commented Oct 17, 2021

beldmit commented Oct 17, 2021

mouse07410 commented Oct 17, 2021 • edited Loading

mouse07410 commented Oct 17, 2021

mouse07410 commented Oct 25, 2021

beldmit commented Oct 26, 2021

beldmit commented Oct 28, 2021

beldmit commented Oct 28, 2021

mouse07410 commented Nov 10, 2021

beldmit commented Nov 14, 2021

mouse07410 commented Nov 14, 2021

beldmit commented Nov 14, 2021

beldmit commented Nov 14, 2021

mouse07410 commented Nov 14, 2021 • edited Loading

beldmit commented Nov 15, 2021

mouse07410 commented Nov 15, 2021 • edited Loading

beldmit commented Nov 15, 2021

mouse07410 commented Nov 15, 2021

mouse07410 commented Oct 17, 2021 •

edited

Loading

mouse07410 commented Nov 14, 2021 •

edited

Loading

mouse07410 commented Nov 15, 2021 •

edited

Loading