Regression metrics

Evaluation platform:

Intel(R) Xeon(R) CPU E5-2695 v4

Summary

full_tool_name	Correct (TP,TN)	Incorrect(FP,FN)	Mix(TP/FN,TN/FP)	Error
Archer 1.0	93	6	2	15
Archer 2.0	104	6	1	5
Archer 2.0*	58	53	0	5
inspector2019-Intel17.0.2	101	9	4	2
inspector2019-Intel18.0.2	104	8	3	1
inspector2019-Intel19.0.0.117	76	39	0	1
inspector2019-Intel19.0.4.227	104	8	4	0
inspector2018-Intel17.0.2	104	10	0	2
inspector2018-Intel18.0.2	105	10	0	1
inspector2018-Intel19.0.0.117	76	37	2	1
inspector2018-Intel19.0.4.227	105	10	1	0
romp-clang8.0.0	99	7	0	10
tsan-5.0.2	97	6	7	6
tsan-6.0.1	100	6	5	5
tsan-7.1.0	97	6	8	5
tsan-8.0.1	94	9	7	6

Metric report:

tool	success rate	error rate	precision-min	precision-max	recall-min	recall-max	accuracy-min	accuracy-max
Archer 1.0	0.87		1	1	0.85	0.89	0.92	0.94
Archer 2.0	0.96		0.52	0.52	0.97	0.97	0.52	0.52
Archer 2.0	0.96		0.98	0.98	0.9	0.91	0.94	0.95
inspector2019-Intel17.0.2	0.98		0.93	0.96	0.85	0.88	0.89	0.92
inspector2019-Intel18.0.2	0.99		0.94	0.96	0.86	0.9	0.9	0.93
inspector2019-Intel19.0.0.117	0.99		0.61	0.61	0.95	0.95	0.66	0.66
inspector2019-Intel19.0.4.227	1		0.93	0.95	0.86	0.92	0.9	0.93
inspector2018-Intel17.0.2	0.98		0.96	0.96	0.86	0.86	0.91	0.91
inspector2018-Intel18.0.2	0.99		0.96	0.96	0.86	0.86	0.91	0.91
inspector2018-Intel19.0.0.117	0.99		0.61	0.62	0.95	0.97	0.66	0.68
inspector2018-Intel19.0.4.227	1		0.94	0.96	0.86	0.86	0.91	0.91
romp-clang8.0.0	0.91		0.96	0.96	0.91	0.91	0.93	0.93
tsan-clang5.0.2-clang5.0.2	0.95		0.98	0.98	0.79	0.91	0.88	0.95
tsan-clang6.0.1-clang6.0.1	0.96		0.98	0.98	0.83	0.91	0.9	0.95
tsan-clang7.1.0-clang7.1.0	0.96		0.96	0.98	0.79	0.91	0.87	0.95
tsan-clang8.0.1-clang8.0.1	0.95		0.96	0.98	0.76	0.86	0.85	0.92
draco	0.4		0.58	0.58	0.55		0.55	0.59

Metrics formula:

precision (P) = TP/(TP + FP)

recall (R) = TP/(TP +FN)

accuracy (A) = (TP +TN)/(TP +FP +TN +FN)

Tool & compiler version:

Tool	Version	Compiler
ThreadSanitizer	8.0.1	Clang 8.0.1
Archer	2.0.0	Clang 6.0.1
Intel Inspector	2019 (build 522981)	Intel 19.0.4.227
romp	6a0ad6d	Clang 8.0.0