Skip to content

Benchmarks 2024 11 26 TVM LLVM O3 spike_rv32

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -O3, Target: spike_rv32 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
13463365.0
( 1.1x )
131158
( 1.398 )
59444
( 3.094 )
0 NCHW TVM Fallback RV32GC 0 -
21001004.0
( 0.7x )
117792
( 1.256 )
59444
( 3.094 )
0 NHWC TVM Fallback RV32GC 0 -
6381992.0
( 2.4x )
127794
( 1.362 )
59444
( 3.094 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
7134191.0
( 2.1x )
128370
( 1.368 )
59444
( 3.094 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8071116.0
( 1.9x )
129220
( 1.377 )
59444
( 3.094 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8436806.0
( 1.8x )
131682
( 1.404 )
59444
( 3.094 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8469966.0
( 1.8x )
132144
( 1.409 )
59444
( 3.094 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8503350.0
( 1.8x )
132680
( 1.414 )
59444
( 3.094 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
3645797.0
( 4.2x )
134166
( 1.43 )
59444
( 3.094 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
2195245.0
( 6.9x )
134084
( 1.429 )
59444
( 3.094 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1458258.0
( 10.4x )
131424
( 1.401 )
59444
( 3.094 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
717026.0
( 21.2x )
121336
( 1.293 )
59444
( 3.094 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
730862.0
( 20.8x )
121454
( 1.295 )
59444
( 3.094 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
730846.0
( 20.8x )
121226
( 1.292 )
59444
( 3.094 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
15202552.0
( Base )
93818
( Base )
19212
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
14560729.0
( 1.0x )
93566
( 0.997 )
23676
( 1.232 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
5735106.0
( 2.7x )
101360
( 1.08 )
19252
( 1.002 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
5154393.0
( 2.9x )
101388
( 1.081 )
19212
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
4862401.0
( 3.1x )
101396
( 1.081 )
19212
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
4723187.0
( 3.2x )
101396
( 1.081 )
19212
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
13192507.0
( 1.2x )
101396
( 1.081 )
19212
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
13185725.0
( 1.2x )
101396
( 1.081 )
19212
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
5108798.0
( 3.0x )
96182
( 1.025 )
23676
( 1.232 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
3974825.0
( 3.8x )
96156
( 1.025 )
23676
( 1.232 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
3416555.0
( 4.4x )
96156
( 1.025 )
23676
( 1.232 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
3375571.0
( 4.5x )
96156
( 1.025 )
23676
( 1.232 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
3378962.0
( 4.5x )
96156
( 1.025 )
23676
( 1.232 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
3382353.0
( 4.5x )
96156
( 1.025 )
23676
( 1.232 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
6472908.0
( 2.3x )
101884
( 1.086 )
23716
( 1.234 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
5595525.0
( 2.7x )
101878
( 1.086 )
23676
( 1.232 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
5303533.0
( 2.9x )
101894
( 1.086 )
23676
( 1.232 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
5164319.0
( 2.9x )
101894
( 1.086 )
23676
( 1.232 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
13632639.0
( 1.1x )
101876
( 1.086 )
23676
( 1.232 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
13629248.0
( 1.1x )
101876
( 1.086 )
23676
( 1.232 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
60473372.0
( 0.9x )
236042
( 1.672 )
108356
( 1.952 )
0 NCHW TVM Fallback RV32GC 0 -
66645204.0
( 0.8x )
231060
( 1.637 )
108356
( 1.952 )
0 NHWC TVM Fallback RV32GC 0 -
115995219.0
( 0.5x )
305918
( 2.167 )
108356
( 1.952 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
115506548.0
( 0.5x )
303592
( 2.151 )
108356
( 1.952 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
114957844.0
( 0.5x )
303884
( 2.153 )
108356
( 1.952 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
114559700.0
( 0.5x )
303030
( 2.147 )
108356
( 1.952 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
115471726.0
( 0.5x )
302414
( 2.142 )
108356
( 1.952 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
116695325.0
( 0.5x )
302278
( 2.141 )
108356
( 1.952 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
54756925.0
( 1.0x )
232944
( 1.65 )
108356
( 1.952 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
51611421.0
( 1.1x )
232800
( 1.649 )
108356
( 1.952 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
49192244.0
( 1.1x )
230352
( 1.632 )
108356
( 1.952 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
47990379.0
( 1.2x )
228872
( 1.621 )
108356
( 1.952 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
47752809.0
( 1.2x )
228656
( 1.62 )
108356
( 1.952 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
47756109.0
( 1.2x )
228316
( 1.617 )
108356
( 1.952 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
56214500.0
( Base )
141166
( Base )
55516
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
72281604.0
( 0.8x )
140742
( 0.997 )
55516
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
11785825.0
( 4.8x )
149886
( 1.062 )
55548
( 1.001 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
11364782.0
( 4.9x )
150026
( 1.063 )
55516
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
14210982.0
( 4.0x )
150030
( 1.063 )
55516
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
15699886.0
( 3.6x )
150030
( 1.063 )
55516
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
16377180.0
( 3.4x )
150030
( 1.063 )
55516
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
35569783.0
( 1.6x )
150030
( 1.063 )
55516
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
15158715.0
( 3.7x )
143790
( 1.019 )
55516
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
9575556.0
( 5.9x )
143918
( 1.019 )
55516
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
7029718.0
( 8.0x )
143918
( 1.019 )
55516
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
5787590.0
( 9.7x )
143918
( 1.019 )
55516
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
4862717.0
( 11.6x )
143918
( 1.019 )
55516
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
4617016.0
( 12.2x )
143918
( 1.019 )
55516
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
17607478.0
( 3.2x )
149168
( 1.057 )
55548
( 1.001 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
16575079.0
( 3.4x )
149304
( 1.058 )
55516
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
17332443.0
( 3.2x )
149312
( 1.058 )
55516
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
17049235.0
( 3.3x )
149312
( 1.058 )
55516
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
16581697.0
( 3.4x )
149312
( 1.058 )
55516
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
43364700.0
( 1.3x )
149312
( 1.058 )
55516
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
1637052.0
( 1.0x )
592646
( 1.864 )
5548
( 1.168 )
0 NCHW TVM Fallback RV32GC 0 -
1637052.0
( 1.0x )
592646
( 1.864 )
5548
( 1.168 )
0 NHWC TVM Fallback RV32GC 0 -
835753.0
( 2.1x )
593666
( 1.867 )
5548
( 1.168 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
776318.0
( 2.2x )
593488
( 1.867 )
5548
( 1.168 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
954212.0
( 1.8x )
593658
( 1.867 )
5548
( 1.168 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
840710.0
( 2.0x )
593476
( 1.867 )
5548
( 1.168 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
817799.0
( 2.1x )
602322
( 1.894 )
5548
( 1.168 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
774304.0
( 2.2x )
594160
( 1.869 )
5548
( 1.168 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
835753.0
( 2.1x )
593666
( 1.867 )
5548
( 1.168 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
776318.0
( 2.2x )
593488
( 1.867 )
5548
( 1.168 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
954212.0
( 1.8x )
593658
( 1.867 )
5548
( 1.168 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
840710.0
( 2.0x )
593476
( 1.867 )
5548
( 1.168 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
817799.0
( 2.1x )
602322
( 1.894 )
5548
( 1.168 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
774304.0
( 2.2x )
594160
( 1.869 )
5548
( 1.168 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1717606.0
( Base )
317940
( Base )
4748
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
1717606.0
( 1.0x )
317940
( 1.0 )
4748
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
584816.0
( 2.9x )
319036
( 1.003 )
4748
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
497567.0
( 3.5x )
318964
( 1.003 )
4748
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
452351.0
( 3.8x )
318964
( 1.003 )
4748
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
429743.0
( 4.0x )
318964
( 1.003 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
418439.0
( 4.1x )
318964
( 1.003 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
1355380.0
( 1.3x )
318964
( 1.003 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
616427.0
( 2.8x )
319246
( 1.004 )
4748
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
499738.0
( 3.4x )
319210
( 1.004 )
4748
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
441398.0
( 3.9x )
319210
( 1.004 )
4748
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
412552.0
( 4.2x )
319210
( 1.004 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
408921.0
( 4.2x )
319210
( 1.004 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
407065.0
( 4.2x )
319210
( 1.004 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
584816.0
( 2.9x )
319036
( 1.003 )
4748
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
497567.0
( 3.5x )
318964
( 1.003 )
4748
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
452351.0
( 3.8x )
318964
( 1.003 )
4748
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
429743.0
( 4.0x )
318964
( 1.003 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
418439.0
( 4.1x )
318964
( 1.003 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
1355380.0
( 1.3x )
318964
( 1.003 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
38499252.0
( 1.2x )
602660
( 1.845 )
180960
( 2.113 )
0 NCHW TVM Fallback RV32GC 0 -
55780162.0
( 0.8x )
577224
( 1.767 )
180960
( 2.113 )
0 NHWC TVM Fallback RV32GC 0 -
72458405.0
( 0.6x )
755390
( 2.313 )
180960
( 2.113 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
71934845.0
( 0.6x )
756492
( 2.316 )
180960
( 2.113 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
72511173.0
( 0.6x )
762182
( 2.333 )
180960
( 2.113 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
72539118.0
( 0.6x )
760590
( 2.329 )
180960
( 2.113 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
72687409.0
( 0.6x )
760480
( 2.328 )
180960
( 2.113 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
71973644.0
( 0.6x )
758496
( 2.322 )
180960
( 2.113 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
12077536.0
( 3.8x )
637022
( 1.95 )
180960
( 2.113 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
8176901.0
( 5.5x )
637512
( 1.952 )
180960
( 2.113 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
6136847.0
( 7.4x )
633342
( 1.939 )
180960
( 2.113 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
5050825.0
( 9.0x )
619146
( 1.896 )
180960
( 2.113 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
4530831.0
( 10.0x )
609480
( 1.866 )
180960
( 2.113 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
4481205.0
( 10.1x )
604794
( 1.852 )
180960
( 2.113 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
45306820.0
( Base )
326632
( Base )
85648
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
44938335.0
( 1.0x )
326518
( 1.0 )
85648
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
17563486.0
( 2.6x )
335238
( 1.026 )
85792
( 1.002 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
17135404.0
( 2.6x )
335370
( 1.027 )
85648
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
18308140.0
( 2.5x )
335378
( 1.027 )
85648
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
21457858.0
( 2.1x )
335378
( 1.027 )
85648
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
24984308.0
( 1.8x )
335378
( 1.027 )
85648
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
38933280.0
( 1.2x )
335376
( 1.027 )
85648
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
13357813.0
( 3.4x )
330080
( 1.011 )
85648
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
10036865.0
( 4.5x )
330190
( 1.011 )
85648
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
8745449.0
( 5.2x )
330190
( 1.011 )
85648
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
8241464.0
( 5.5x )
330188
( 1.011 )
85648
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
8197090.0
( 5.5x )
330192
( 1.011 )
85648
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
8200480.0
( 5.5x )
330188
( 1.011 )
85648
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
20382369.0
( 2.2x )
335900
( 1.028 )
85792
( 1.002 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
19256004.0
( 2.4x )
335996
( 1.029 )
85648
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
20767428.0
( 2.2x )
336010
( 1.029 )
85648
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
23917145.0
( 1.9x )
336008
( 1.029 )
85648
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
27435266.0
( 1.7x )
335994
( 1.029 )
85648
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
41387629.0
( 1.1x )
335994
( 1.029 )
85648
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally