Benchmarks 2024 11 26 TFLM LLVM Os spike_rv64

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)
  • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
  • Spike PK : fix-gcc14-rvv

Toolchains

• RISC-V GCC:

  • Scalar: riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118
  • Vector: riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118
  • Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257

• LLVM/Clang:

  • clang version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
  • Linker: lld (TODO)

Models

• MLPerfTiny Benchmark

• TODO: others!

Frameworks

• MLonMCU : develop

• TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -Os, Target: spike_rv64 )

Audio Wake Words (aww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
39715465.0 ( 0.4x )	145894 ( 0.867 )	38312 ( 1.0 )	0	TFLM	Reference	RV64GC	0	-
31995165.0 ( 0.5x )	151022 ( 0.898 )	38320 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
30079482.0 ( 0.5x )	150948 ( 0.897 )	38320 ( 1.0 )	256	TFLM	Reference	RV64GCV	0	Loop+SLP
29113993.0 ( 0.5x )	151156 ( 0.899 )	38320 ( 1.0 )	512	TFLM	Reference	RV64GCV	0	Loop+SLP
28630598.0 ( 0.5x )	151466 ( 0.9 )	38320 ( 1.0 )	1024	TFLM	Reference	RV64GCV	0	Loop+SLP
28630607.0 ( 0.5x )	151670 ( 0.902 )	38320 ( 1.0 )	2048	TFLM	Reference	RV64GCV	0	Loop+SLP
28644647.0 ( 0.5x )	173666 ( 1.032 )	38320 ( 1.0 )	4096	TFLM	Reference	RV64GCV	0	Loop+SLP
14708091.0 ( Base )	168210 ( Base )	38312 ( Base )	0	muRISCV-NN	Scalar	RV64GC	0	-
15078549.0 ( 1.0x )	167462 ( 0.996 )	38312 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV64GC	0	-
5820243.0 ( 2.5x )	176044 ( 1.047 )	38320 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
4782120.0 ( 3.1x )	175966 ( 1.046 )	38320 ( 1.0 )	256	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
4265038.0 ( 3.4x )	175990 ( 1.046 )	38320 ( 1.0 )	512	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
3964941.0 ( 3.7x )	176216 ( 1.048 )	38320 ( 1.0 )	1024	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
3964944.0 ( 3.7x )	176310 ( 1.048 )	38320 ( 1.0 )	2048	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
3970551.0 ( 3.7x )	188674 ( 1.122 )	38320 ( 1.0 )	4096	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
4074938.0 ( 3.6x )	169684 ( 1.009 )	38312 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-
2819922.0 ( 5.2x )	169542 ( 1.008 )	38312 ( 1.0 )	256	muRISCV-NN	Vector	RV64GCV	0	-
2142586.0 ( 6.9x )	169414 ( 1.007 )	38312 ( 1.0 )	512	muRISCV-NN	Vector	RV64GCV	0	-
2103455.0 ( 7.0x )	169444 ( 1.007 )	38312 ( 1.0 )	1024	muRISCV-NN	Vector	RV64GCV	0	-
2103455.0 ( 7.0x )	169444 ( 1.007 )	38312 ( 1.0 )	2048	muRISCV-NN	Vector	RV64GCV	0	-
2105320.0 ( 7.0x )	169444 ( 1.007 )	38312 ( 1.0 )	4096	muRISCV-NN	Vector	RV64GCV	0	-
6910795.0 ( 2.1x )	174910 ( 1.04 )	38320 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5905744.0 ( 2.5x )	174832 ( 1.039 )	38320 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5403333.0 ( 2.7x )	174856 ( 1.04 )	38320 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5121769.0 ( 2.9x )	175082 ( 1.041 )	38320 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5116646.0 ( 2.9x )	175176 ( 1.041 )	38320 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5127377.0 ( 2.9x )	187540 ( 1.115 )	38320 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP

Image Classification (resnet)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
121110788.0 ( 0.5x )	187306 ( 0.928 )	71080 ( 1.0 )	0	TFLM	Reference	RV64GC	0	-
57760563.0 ( 1.0x )	193368 ( 0.958 )	71088 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
47307258.0 ( 1.2x )	193366 ( 0.958 )	71088 ( 1.0 )	256	TFLM	Reference	RV64GCV	0	Loop+SLP
44797070.0 ( 1.2x )	193794 ( 0.96 )	71088 ( 1.0 )	512	TFLM	Reference	RV64GCV	0	Loop+SLP
44334667.0 ( 1.3x )	194232 ( 0.962 )	71088 ( 1.0 )	1024	TFLM	Reference	RV64GCV	0	Loop+SLP
44324043.0 ( 1.3x )	194556 ( 0.964 )	71088 ( 1.0 )	2048	TFLM	Reference	RV64GCV	0	Loop+SLP
44344333.0 ( 1.3x )	216082 ( 1.07 )	71088 ( 1.0 )	4096	TFLM	Reference	RV64GCV	0	Loop+SLP
55734444.0 ( Base )	201896 ( Base )	71080 ( Base )	0	muRISCV-NN	Scalar	RV64GC	0	-
71563966.0 ( 0.8x )	201358 ( 0.997 )	71080 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV64GC	0	-
25739743.0 ( 2.2x )	210808 ( 1.044 )	71088 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
17776724.0 ( 3.1x )	210736 ( 1.044 )	71088 ( 1.0 )	256	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
14051138.0 ( 4.0x )	211024 ( 1.045 )	71088 ( 1.0 )	512	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
12355748.0 ( 4.5x )	211402 ( 1.047 )	71088 ( 1.0 )	1024	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
11526338.0 ( 4.8x )	211640 ( 1.048 )	71088 ( 1.0 )	2048	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
10900009.0 ( 5.1x )	228720 ( 1.133 )	71088 ( 1.0 )	4096	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
15246284.0 ( 3.7x )	204114 ( 1.011 )	71080 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-
9663084.0 ( 5.8x )	203982 ( 1.01 )	71080 ( 1.0 )	256	muRISCV-NN	Vector	RV64GCV	0	-
7117244.0 ( 7.8x )	203854 ( 1.01 )	71080 ( 1.0 )	512	muRISCV-NN	Vector	RV64GCV	0	-
5876981.0 ( 9.5x )	203884 ( 1.01 )	71080 ( 1.0 )	1024	muRISCV-NN	Vector	RV64GCV	0	-
4948717.0 ( 11.3x )	203884 ( 1.01 )	71080 ( 1.0 )	2048	muRISCV-NN	Vector	RV64GCV	0	-
4701490.0 ( 11.9x )	203884 ( 1.01 )	71080 ( 1.0 )	4096	muRISCV-NN	Vector	RV64GCV	0	-
19843671.0 ( 2.8x )	209586 ( 1.038 )	71088 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
13551429.0 ( 4.1x )	209514 ( 1.038 )	71088 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
10601906.0 ( 5.3x )	209802 ( 1.039 )	71088 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
9260268.0 ( 6.0x )	210180 ( 1.041 )	71088 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
8610061.0 ( 6.5x )	210418 ( 1.042 )	71088 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
8124607.0 ( 6.9x )	227498 ( 1.127 )	71088 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
2781498.0 ( 0.6x )	340870 ( 0.984 )	21448 ( 1.0 )	0	TFLM	Reference	RV64GC	0	-
957732.0 ( 1.7x )	343230 ( 0.99 )	21448 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
694594.0 ( 2.4x )	343080 ( 0.99 )	21448 ( 1.0 )	256	TFLM	Reference	RV64GCV	0	Loop+SLP
563040.0 ( 2.9x )	343094 ( 0.99 )	21448 ( 1.0 )	512	TFLM	Reference	RV64GCV	0	Loop+SLP
497278.0 ( 3.3x )	343308 ( 0.991 )	21448 ( 1.0 )	1024	TFLM	Reference	RV64GCV	0	Loop+SLP
459286.0 ( 3.6x )	343392 ( 0.991 )	21448 ( 1.0 )	2048	TFLM	Reference	RV64GCV	0	Loop+SLP
467540.0 ( 3.5x )	350760 ( 1.012 )	21448 ( 1.0 )	4096	TFLM	Reference	RV64GCV	0	Loop+SLP
1632483.0 ( Base )	346574 ( Base )	21448 ( Base )	0	muRISCV-NN	Scalar	RV64GC	0	-
2978070.0 ( 0.5x )	346578 ( 1.0 )	21448 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV64GC	0	-
593018.0 ( 2.8x )	350270 ( 1.011 )	21448 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
475576.0 ( 3.4x )	350096 ( 1.01 )	21448 ( 1.0 )	256	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
416854.0 ( 3.9x )	350050 ( 1.01 )	21448 ( 1.0 )	512	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
387484.0 ( 4.2x )	350228 ( 1.011 )	21448 ( 1.0 )	1024	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
372818.0 ( 4.4x )	350276 ( 1.011 )	21448 ( 1.0 )	2048	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
376094.0 ( 4.3x )	355450 ( 1.026 )	21448 ( 1.0 )	4096	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
1935621.0 ( 0.8x )	348202 ( 1.005 )	21448 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-
1817253.0 ( 0.9x )	348096 ( 1.004 )	21448 ( 1.0 )	256	muRISCV-NN	Vector	RV64GCV	0	-
1758068.0 ( 0.9x )	347974 ( 1.004 )	21448 ( 1.0 )	512	muRISCV-NN	Vector	RV64GCV	0	-
1728813.0 ( 0.9x )	348004 ( 1.004 )	21448 ( 1.0 )	1024	muRISCV-NN	Vector	RV64GCV	0	-
1725129.0 ( 0.9x )	348004 ( 1.004 )	21448 ( 1.0 )	2048	muRISCV-NN	Vector	RV64GCV	0	-
1723245.0 ( 0.9x )	348004 ( 1.004 )	21448 ( 1.0 )	4096	muRISCV-NN	Vector	RV64GCV	0	-
992695.0 ( 1.6x )	350274 ( 1.011 )	21448 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
694330.0 ( 2.4x )	350100 ( 1.01 )	21448 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
545142.0 ( 3.0x )	350054 ( 1.01 )	21448 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
470546.0 ( 3.5x )	350232 ( 1.011 )	21448 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
433245.0 ( 3.8x )	350280 ( 1.011 )	21448 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
433723.0 ( 3.8x )	355454 ( 1.026 )	21448 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
104294920.0 ( 0.4x )	419536 ( 0.949 )	136616 ( 1.0 )	0	TFLM	Reference	RV64GC	0	-
67375214.0 ( 0.7x )	424664 ( 0.961 )	136624 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
61850250.0 ( 0.7x )	424590 ( 0.961 )	136624 ( 1.0 )	256	TFLM	Reference	RV64GCV	0	Loop+SLP
59218368.0 ( 0.7x )	424798 ( 0.961 )	136624 ( 1.0 )	512	TFLM	Reference	RV64GCV	0	Loop+SLP
58109096.0 ( 0.8x )	425108 ( 0.962 )	136624 ( 1.0 )	1024	TFLM	Reference	RV64GCV	0	Loop+SLP
57659751.0 ( 0.8x )	425312 ( 0.963 )	136624 ( 1.0 )	2048	TFLM	Reference	RV64GCV	0	Loop+SLP
57638766.0 ( 0.8x )	447076 ( 1.012 )	136624 ( 1.0 )	4096	TFLM	Reference	RV64GCV	0	Loop+SLP
43982864.0 ( Base )	441852 ( Base )	136616 ( Base )	0	muRISCV-NN	Scalar	RV64GC	0	-
45000202.0 ( 1.0x )	441104 ( 0.998 )	136616 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV64GC	0	-
18369396.0 ( 2.4x )	449686 ( 1.018 )	136624 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
15392478.0 ( 2.9x )	449608 ( 1.018 )	136624 ( 1.0 )	256	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13968194.0 ( 3.1x )	449632 ( 1.018 )	136624 ( 1.0 )	512	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13442124.0 ( 3.3x )	449858 ( 1.018 )	136624 ( 1.0 )	1024	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13226523.0 ( 3.3x )	449952 ( 1.018 )	136624 ( 1.0 )	2048	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13213856.0 ( 3.3x )	462084 ( 1.046 )	136624 ( 1.0 )	4096	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13376084.0 ( 3.3x )	443326 ( 1.003 )	136616 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-
10055028.0 ( 4.4x )	443184 ( 1.003 )	136616 ( 1.0 )	256	muRISCV-NN	Vector	RV64GCV	0	-
8763584.0 ( 5.0x )	443056 ( 1.003 )	136616 ( 1.0 )	512	muRISCV-NN	Vector	RV64GCV	0	-
8261445.0 ( 5.3x )	443086 ( 1.003 )	136616 ( 1.0 )	1024	muRISCV-NN	Vector	RV64GCV	0	-
8213670.0 ( 5.4x )	443086 ( 1.003 )	136616 ( 1.0 )	2048	muRISCV-NN	Vector	RV64GCV	0	-
8215535.0 ( 5.4x )	443086 ( 1.003 )	136616 ( 1.0 )	4096	muRISCV-NN	Vector	RV64GCV	0	-
21985474.0 ( 2.0x )	448552 ( 1.015 )	136624 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
19086581.0 ( 2.3x )	448474 ( 1.015 )	136624 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
17699465.0 ( 2.5x )	448498 ( 1.015 )	136624 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
17175117.0 ( 2.6x )	448724 ( 1.016 )	136624 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
16959528.0 ( 2.6x )	448818 ( 1.016 )	136624 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
16957090.0 ( 2.6x )	460950 ( 1.043 )	136624 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP

Original data

Click here to download the raw files for this benchmark.