Skip to content

Benchmarks 2024 11 26 TFLM LLVM Os spike_rv32_min

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -Os, Target: spike_rv32_min )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
39232601.0
( 0.4x )
202416
( 0.856 )
35992
( 1.0 )
0 TFLM Reference RV32IM 0 -
33275950.0
( 0.5x )
208396
( 0.882 )
35992
( 1.0 )
128 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
31483286.0
( 0.5x )
208380
( 0.882 )
35992
( 1.0 )
256 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
208684
( 0.883 )
35992
( 1.0 )
512 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
30133659.0
( 0.5x )
209024
( 0.884 )
35992
( 1.0 )
1024 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
30134089.0
( 0.5x )
209228
( 0.885 )
35992
( 1.0 )
2048 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
30134050.0
( 0.5x )
209432
( 0.886 )
35992
( 1.0 )
4096 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
15024169.0
( Base )
236344
( Base )
35992
( Base )
0 muRISCV-NN Scalar RV32IM 0 -
14914746.0
( 1.0x )
235156
( 0.995 )
35992
( 1.0 )
0 muRISCV-NN Vector (Portable) RV32IM 0 -
6087854.0
( 2.5x )
246468
( 1.043 )
35992
( 1.0 )
128 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
5081221.0
( 3.0x )
246416
( 1.043 )
35992
( 1.0 )
256 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
4581047.0
( 3.3x )
246552
( 1.043 )
35992
( 1.0 )
512 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
4288945.0
( 3.5x )
246776
( 1.044 )
35992
( 1.0 )
1024 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
4288852.0
( 3.5x )
246872
( 1.045 )
35992
( 1.0 )
2048 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
4286501.0
( 3.5x )
246968
( 1.045 )
35992
( 1.0 )
4096 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
4103590.0
( 3.7x )
237520
( 1.005 )
35992
( 1.0 )
128 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2848612.0
( 5.3x )
237412
( 1.005 )
35992
( 1.0 )
256 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2166149.0
( 6.9x )
237408
( 1.005 )
35992
( 1.0 )
512 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2130388.0
( 7.1x )
237460
( 1.005 )
35992
( 1.0 )
1024 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2130809.0
( 7.1x )
237460
( 1.005 )
35992
( 1.0 )
2048 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2131134.0
( 7.0x )
237460
( 1.005 )
35992
( 1.0 )
4096 muRISCV-NN Vector RV32IM_ZVE64X 0 -
6842130.0
( 2.2x )
245108
( 1.037 )
35992
( 1.0 )
128 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
5580208.0
( 2.7x )
244920
( 1.036 )
35992
( 1.0 )
256 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
5097433.0
( 2.9x )
245056
( 1.037 )
35992
( 1.0 )
512 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
245280
( 1.038 )
35992
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
4824122.0
( 3.1x )
245376
( 1.038 )
35992
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
4824887.0
( 3.1x )
245472
( 1.039 )
35992
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
120988348.0
( 0.5x )
245320
( 0.916 )
68760
( 1.0 )
0 TFLM Reference RV32IM 0 -
-
( ?x )
251104
( 0.937 )
68760
( 1.0 )
128 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
251176
( 0.937 )
68760
( 1.0 )
256 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
251680
( 0.939 )
68760
( 1.0 )
512 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
252140
( 0.941 )
68760
( 1.0 )
1024 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
252464
( 0.942 )
68760
( 1.0 )
2048 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
252788
( 0.943 )
68760
( 1.0 )
4096 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
56266438.0
( Base )
267936
( Base )
68760
( Base )
0 muRISCV-NN Scalar RV32IM 0 -
72492194.0
( 0.8x )
267072
( 0.997 )
68760
( 1.0 )
0 muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
278612
( 1.04 )
68760
( 1.0 )
128 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
278616
( 1.04 )
68760
( 1.0 )
256 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
278992
( 1.041 )
68760
( 1.0 )
512 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
279360
( 1.043 )
68760
( 1.0 )
1024 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
279600
( 1.044 )
68760
( 1.0 )
2048 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
279840
( 1.044 )
68760
( 1.0 )
4096 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
15270545.0
( 3.7x )
270228
( 1.009 )
68760
( 1.0 )
128 muRISCV-NN Vector RV32IM_ZVE64X 0 -
9687783.0
( 5.8x )
270128
( 1.008 )
68760
( 1.0 )
256 muRISCV-NN Vector RV32IM_ZVE64X 0 -
7142289.0
( 7.9x )
270124
( 1.008 )
68760
( 1.0 )
512 muRISCV-NN Vector RV32IM_ZVE64X 0 -
5900248.0
( 9.5x )
270176
( 1.008 )
68760
( 1.0 )
1024 muRISCV-NN Vector RV32IM_ZVE64X 0 -
4972641.0
( 11.3x )
270176
( 1.008 )
68760
( 1.0 )
2048 muRISCV-NN Vector RV32IM_ZVE64X 0 -
4721620.0
( 11.9x )
270176
( 1.008 )
68760
( 1.0 )
4096 muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
277008
( 1.034 )
68760
( 1.0 )
128 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
277012
( 1.034 )
68760
( 1.0 )
256 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
277388
( 1.035 )
68760
( 1.0 )
512 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
277756
( 1.037 )
68760
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
277996
( 1.038 )
68760
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
278236
( 1.038 )
68760
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
2797784.0
( 0.6x )
384944
( 0.979 )
19336
( 1.0 )
0 TFLM Reference RV32IM 0 -
-
( ?x )
387420
( 0.985 )
19336
( 1.0 )
128 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
387320
( 0.985 )
19336
( 1.0 )
256 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
387452
( 0.985 )
19336
( 1.0 )
512 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
387672
( 0.986 )
19336
( 1.0 )
1024 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
387756
( 0.986 )
19336
( 1.0 )
2048 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
387840
( 0.986 )
19336
( 1.0 )
4096 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
1700249.0
( Base )
393160
( Base )
19336
( Base )
0 muRISCV-NN Scalar RV32IM 0 -
3052881.0
( 0.6x )
393164
( 1.0 )
19336
( 1.0 )
0 muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
397540
( 1.011 )
19336
( 1.0 )
128 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397412
( 1.011 )
19336
( 1.0 )
256 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397488
( 1.011 )
19336
( 1.0 )
512 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397672
( 1.011 )
19336
( 1.0 )
1024 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397720
( 1.012 )
19336
( 1.0 )
2048 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397768
( 1.012 )
19336
( 1.0 )
4096 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
1945586.0
( 0.9x )
395372
( 1.006 )
19336
( 1.0 )
128 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1828906.0
( 0.9x )
395276
( 1.005 )
19336
( 1.0 )
256 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1770566.0
( 1.0x )
395284
( 1.005 )
19336
( 1.0 )
512 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1741720.0
( 1.0x )
395344
( 1.006 )
19336
( 1.0 )
1024 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1738089.0
( 1.0x )
395344
( 1.006 )
19336
( 1.0 )
2048 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1736233.0
( 1.0x )
395344
( 1.006 )
19336
( 1.0 )
4096 muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
397544
( 1.011 )
19336
( 1.0 )
128 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397416
( 1.011 )
19336
( 1.0 )
256 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397492
( 1.011 )
19336
( 1.0 )
512 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397676
( 1.011 )
19336
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397724
( 1.012 )
19336
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
397772
( 1.012 )
19336
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
103372570.0
( 0.4x )
476056
( 0.933 )
134296
( 1.0 )
0 TFLM Reference RV32IM 0 -
-
( ?x )
482036
( 0.945 )
134296
( 1.0 )
128 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
482020
( 0.945 )
134296
( 1.0 )
256 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
482324
( 0.946 )
134296
( 1.0 )
512 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
482664
( 0.946 )
134296
( 1.0 )
1024 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
482868
( 0.947 )
134296
( 1.0 )
2048 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
483072
( 0.947 )
134296
( 1.0 )
4096 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
44946617.0
( Base )
509984
( Base )
134296
( Base )
0 muRISCV-NN Scalar RV32IM 0 -
44831241.0
( 1.0x )
508796
( 0.998 )
134296
( 1.0 )
0 muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
520108
( 1.02 )
134296
( 1.0 )
128 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
520056
( 1.02 )
134296
( 1.0 )
256 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
520192
( 1.02 )
134296
( 1.0 )
512 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
520416
( 1.02 )
134296
( 1.0 )
1024 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
520512
( 1.021 )
134296
( 1.0 )
2048 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
520608
( 1.021 )
134296
( 1.0 )
4096 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
13391092.0
( 3.4x )
511160
( 1.002 )
134296
( 1.0 )
128 muRISCV-NN Vector RV32IM_ZVE64X 0 -
10070110.0
( 4.5x )
511052
( 1.002 )
134296
( 1.0 )
256 muRISCV-NN Vector RV32IM_ZVE64X 0 -
8778682.0
( 5.1x )
511048
( 1.002 )
134296
( 1.0 )
512 muRISCV-NN Vector RV32IM_ZVE64X 0 -
8274774.0
( 5.4x )
511100
( 1.002 )
134296
( 1.0 )
1024 muRISCV-NN Vector RV32IM_ZVE64X 0 -
8227414.0
( 5.5x )
511100
( 1.002 )
134296
( 1.0 )
2048 muRISCV-NN Vector RV32IM_ZVE64X 0 -
8227733.0
( 5.5x )
511100
( 1.002 )
134296
( 1.0 )
4096 muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
518748
( 1.017 )
134296
( 1.0 )
128 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
518560
( 1.017 )
134296
( 1.0 )
256 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
518696
( 1.017 )
134296
( 1.0 )
512 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
518920
( 1.018 )
134296
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
519016
( 1.018 )
134296
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
519112
( 1.018 )
134296
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally