Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

brg:sve_256 fails benchdnn accuracy tests #2008

Open
jondea opened this issue Jul 24, 2024 · 0 comments
Open

brg:sve_256 fails benchdnn accuracy tests #2008

jondea opened this issue Jul 24, 2024 · 0 comments
Labels
bug A confirmed library bug help wanted platform:cpu-aarch64 Codeowner: @oneapi-src/onednn-cpu-aarch64

Comments

@jondea
Copy link
Contributor

jondea commented Jul 24, 2024

Summary

test_benchdnn_modeC_matmul_multidims_cpu is failing due to brg:sve_256 on main/v3.6 319a77e

The failing benchdnn calls are

> ./tests/benchdnn/benchdnn --matmul --stag=acbd --wtag=acbd --dtag=acbd --attr-fpmath=bf16, 1x16x38x64:105x1x64x38
[23104][DST][1:0:0:0] exp_f32:         -71 exp:         -71 got:        -157 diff:      86 rdiff: 1.21127
[23105][DST][1:0:0:1] exp_f32:          15 exp:          15 got:           8 diff:       7 rdiff:0.466667
[23106][DST][1:0:0:2] exp_f32:         186 exp:         186 got:        -129 diff:     315 rdiff: 1.69355
[23107][DST][1:0:0:3] exp_f32:          -3 exp:          -3 got:        -100 diff:      97 rdiff: 32.3333
[23108][DST][1:0:0:4] exp_f32:          -1 exp:          -1 got:          28 diff:      29 rdiff:      29
[23109][DST][1:0:0:5] exp_f32:         100 exp:         100 got:          40 diff:      60 rdiff:     0.6
[23110][DST][1:0:0:6] exp_f32:          15 exp:          15 got:         -77 diff:      92 rdiff: 6.13333
[23111][DST][1:0:0:7] exp_f32:          95 exp:          95 got:         -44 diff:     139 rdiff: 1.46316
[23112][DST][1:0:0:8] exp_f32:         -62 exp:         -62 got:         106 diff:     168 rdiff: 2.70968
[23113][DST][1:0:0:9] exp_f32:          31 exp:          31 got:         117 diff:      86 rdiff: 2.77419
[COMPARE_STATS][DST]: trh=1e-06 err_max_diff:     815 err_max_rdiff:     435 all_max_diff:     815 all_max_rdiff:     435
0:FAILED (errors:2396156 total:2425920) __REPRO: --matmul --stag=acbd --wtag=acbd --dtag=acbd --attr-fpmath=bf16 1x16x38x64:105x1x64x38
[23104][DST][1:0:0:0] exp_f32:      -15580 exp:      -15580 got:      -34229 diff:   18649 rdiff: 1.19698
[23105][DST][1:0:0:1] exp_f32:        7185 exp:        7185 got:       -1041 diff:    8226 rdiff: 1.14489
[23106][DST][1:0:0:2] exp_f32:       41612 exp:       41612 got:      -27123 diff:   68735 rdiff: 1.65181
[23107][DST][1:0:0:3] exp_f32:        2969 exp:        2969 got:      -15306 diff:   18275 rdiff: 6.15527
[23108][DST][1:0:0:4] exp_f32:        3112 exp:        3112 got:        8814 diff:    5702 rdiff: 1.83226
[23109][DST][1:0:0:5] exp_f32:       16910 exp:       16910 got:       12855 diff:    4055 rdiff:0.239799
[23110][DST][1:0:0:6] exp_f32:        -401 exp:        -401 got:      -12707 diff:   12306 rdiff: 30.6883
[23111][DST][1:0:0:7] exp_f32:       25251 exp:       25251 got:      -10682 diff:   35933 rdiff: 1.42303
[23112][DST][1:0:0:8] exp_f32:      -12964 exp:      -12964 got:       23692 diff:   36656 rdiff: 2.82752
[23113][DST][1:0:0:9] exp_f32:        5708 exp:        5708 got:       24813 diff:   19105 rdiff: 3.34706
[COMPARE_STATS][DST]: trh=1e-06 err_max_diff:  176270 err_max_rdiff:   60963 all_max_diff:  176270 all_max_rdiff:   60963
1:FAILED (errors:2402790 total:2425920) __REPRO: --matmul --stag=acbd --wtag=acbd --dtag=acbd 1x16x38x64:105x1x64x38
tests:2 passed:0 skipped:0 mistrusted:0 unimplemented:0 invalid_arguments:0 failed:2 listed:0

Pinging @vineelabhinav @Shreyas-fuj from git blame

Environment

  • CPU: Neoverse-V1 C7g.4xlarge
  • OS version: Ubuntu 20.04
  • Compiler version: gcc-10
  • git hash: 319a77e
  • CMake version: 3.16.3
  • CMake output log
+ cmake -DDNNL_CPU_RUNTIME=OMP -DCMAKE_BUILD_TYPE=Release -DDNNL_BUILD_FOR_CI=ON -DDNNL_WERROR=OFF -DDNNL_TEST_SET=NIGHTLY ..
-- The C compiler identification is GNU 10.5.0
-- The CXX compiler identification is GNU 10.5.0
-- Check for working C compiler: /usr/bin/gcc-10
-- Check for working C compiler: /usr/bin/gcc-10 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++-10
-- Check for working CXX compiler: /usr/bin/g++-10 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- DNNL_TARGET_ARCH: AARCH64
-- DNNL_LIBRARY_NAME: dnnl
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Could NOT find Doxyrest (missing: DOXYREST_EXECUTABLE)
-- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.18", minimum required is "2.7")
-- Found Sphinx: /usr/bin/sphinx-build (found version "sphinx-build 1.8.5")
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Enabled testing coverage: NIGHTLY
-- Enabled workload: TRAINING
-- Enabled primitives: ALL
-- Enabled primitive CPU ISA: ALL
-- Enabled primitive GPU ISA: ALL
-- Enabled GeMM kernels ISA: ALL
-- Primitive cache is enabled
-- Graph component is enabled
-- Graph compiler backend is disabled.
-- Configuring done
-- Generating done
-- Build files have been written to: oneDNN/build
@jondea jondea added sighting Suspicious library behavior. Should be promoted to a bug when confirmed platform:cpu-aarch64 Codeowner: @oneapi-src/onednn-cpu-aarch64 labels Jul 24, 2024
@vpirogov vpirogov added help wanted bug A confirmed library bug and removed sighting Suspicious library behavior. Should be promoted to a bug when confirmed labels Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A confirmed library bug help wanted platform:cpu-aarch64 Codeowner: @oneapi-src/onednn-cpu-aarch64
Projects
None yet
Development

No branches or pull requests

2 participants