Benchmarks 2023 04 17

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

• Scalar: Self compiled on master branch of riscv-gnu-toolchain
• Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
• Vector: Self compiled using this working-branch: riscv-gcc-rvv-next

Models

• MLPerfTiny Benchmark

Package Versions

• MLonMCU : 2ab749d3747c04f3b01973ef955bda5be0964f0f
• TFLM : f050eec7e32a0895f7658db21a4bdbd0975087a5
• TVM : Nightly Pre-Build
• Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
• Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Frameworks

Benchmarks exist for integration with TensorFlow Lite for Microcontrollers (TFLM) and TVM.

TFLite for Microcontrollers

Comparing reference kernel implementations (Default) with cmsis-nn optimized kernels and linkage with libmuriscvnn.a.

Audio Wake Words (aww)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
153529267	144687	36192	63653	79826	0	Default
15509829	158148	36200	63656	93284	0	muRISCV-NN
9026170	157944	36208	63652	93088	64	muRISCV-NN	VEXT
2135130	157944	36208	63652	93088	1024	muRISCV-NN	VEXT
12172435	196730	36344	70740	120994	0	muRISCV-NN	PEXT

Visual Wake Words (vww)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
433523791	418411	134508	337293	79910	0	Default
46510032	431872	134516	337296	93368	0	muRISCV-NN
27212603	431668	134524	337292	93172	64	muRISCV-NN	VEXT
8775554	431668	134524	337292	93172	1024	muRISCV-NN	VEXT
38396156	470446	134660	344380	121070	0	muRISCV-NN	PEXT

Image Classification (resnet)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
687851073	184423	68956	102485	80730	0	Default
62519499	192280	68948	102488	88584	0	muRISCV-NN
37856218	192866	68956	102500	89162	64	muRISCV-NN	VEXT
6058820	192866	68956	102500	89162	1024	muRISCV-NN	VEXT
28124855	233826	69092	109572	119258	0	muRISCV-NN	PEXT

Anomaly Detection (toycar)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
3434813	347340	19392	280872	65260	0	Default
1920500	349928	19392	280872	67848	0	muRISCV-NN
729214	349838	19396	280868	67766	64	muRISCV-NN	VEXT
538790	349838	19396	280868	67766	1024	muRISCV-NN	VEXT
1174689	389048	19536	287956	96096	0	muRISCV-NN	PEXT

TVM

Using CMSIS-NN BYOC feature and linkage with libmuriscvnn.a.

Extension mapping:

• VEXT -> cortex-m55
• PEXT -> cortex-m7

Used data/kernel layouts:

• Untuned: NHWC (TFLite default)
• Tuned: NCHW (Converted using TVM)

Audio Wake Words (aww)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Layout	Kernels	Extensions
31435272	117272	59448	56212	61044	0	NCHW	Default
29580732	111594	59448	56212	55366	0	NHWC	Default
15699726	110750	51276	56212	54522	0	NCHW	Autotuned
29576111	111724	59448	56212	55496	0	NHWC	Autotuned
15355980	99626	19140	36644	62966	0		muRISCV-NN
10091110	99576	23604	36644	62916	64		muRISCV-NN	VEXT
3987654	99576	23604	36644	62916	1024		muRISCV-NN	VEXT
12834237	101442	20292	36400	64966	0		muRISCV-NN	PEXT

Visual Wake Words (vww)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Layout	Kernels	Extensions
90828165	550218	180968	462932	87270	0	NCHW	Default
86102169	530776	180968	462932	67828	0	NHWC	Default
46356353	531550	180968	462932	68602	0	NCHW	Autotuned
86102172	530780	180968	462932	67832	0	NHWC	Autotuned
45331325	333214	85588	266324	66874	0		muRISCV-NN
26022364	333452	85588	266324	67112	64		muRISCV-NN	VEXT
7585315	333472	85588	266324	67132	1024		muRISCV-NN	VEXT
37187229	335288	85588	266080	69132	0		muRISCV-NN	PEXT

Image Classification (resnet)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Layout	Kernels	Extensions
138976677	226522	108360	163156	63350	0	NCHW	Default
115540068	217372	108360	163156	54200	0	NHWC	Default
56900335	220452	92176	163156	57280	0	NCHW	Autotuned
115536141	217470	108360	163156	54298	0	NHWC	Autotuned
62228686	146938	55444	87332	59590	0		muRISCV-NN
37562250	147640	55444	87332	60292	64		muRISCV-NN	VEXT
5764852	147640	55444	87332	60292	1024		muRISCV-NN	VEXT
27817070	151786	55444	87088	64622	0		muRISCV-NN	PEXT

Anomaly Detection (toycar)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Layout	Kernels	Extensions
3134840	589618	5492	537988	51614	0	NCHW	Default
3134840	589618	5492	537988	51614	0	NHWC	Default
2041108	617982	6804	537988	79978	0	NCHW	Autotuned
2041108	617982	6804	537988	79978	0	NHWC	Autotuned
1822341	324918	4692	273924	50978	0		muRISCV-NN
626114	324966	4692	273924	51026	64		muRISCV-NN	VEXT
435688	324962	4692	273924	51022	1024		muRISCV-NN	VEXT
1066406	327242	4692	273680	53486	0		muRISCV-NN	PEXT

Code size compared to CMSIS-NN

WIP