Metrics Aug 2020

Evaluation platform:

CPU Architecture - IBM Power9 and Intel Xeon

GPU Architecture - NVIDIA V100 (Volta)

Tool	Languages	TP	FP	TN	FN	Recall	Specificity	Precision	Accuracy	TSR	Adjusted F1
Archer	C/C++	63	1	80	17	0.7875	0.9877	0.9844	0.8882	0.9360	0.8190
Intel Inspector	C/C++	71	40	45	8	0.8987	0.5294	0.6396	0.7073	0.9535	0.7126
ROMP	C/C++	59	11	73	18	0.7590	0.8876	0.8630	0.8256	1	0.8077
ThreadSanitizer	C/C++	65	1	86	16	0.8025	0.9885	0.9848	0.8988	0.9767	0.8638
Archer	Fortran	52	4	58	16	0.7647	0.9355	0.9286	0.8462	0.7831	0.6568
Intel Inspector	Fortran	61	13	62	20	0.7531	0.8267	0.8243	0.7885	0.9398	0.7397
ROMP	Fortran	57	13	61	22	0.7215	0.8243	0.8143	0.7712	0.9217	0.7052
ThreadSanitizer	Fortran	55	36	29	12	0.8209	0.4462	0.6044	0.6364	0.7952	0.5536

precision (P) = TP/(TP + FP)

recall (R) = TP/(TP +FN)

accuracy (A) = (TP +TN)/(TP +FP +TN +FN)

specificity (S) = TN / ( TN + FP)

F1 score (F1) = 2 * (P * R) / (P + R)

test support rate (TSR) = (TP +FP +TN +FN)/(TEST NUMBER)

adjusted F1 (AF1) = TSR * F1