Regression metrics

Evaluation platform:

Intel(R) Xeon(R) CPU E5-2695 v4

Summary

Tool-Compiler	Correct (TP/TN)	Incorrect (FP/FN)	Mix	Error
archer1.0-clang3.9.1	93	6	2	15
archer2.0-clang6.0.0	58	53	0	5
archer2.0-clang6.0.0*	104	6	1	5
inspector2018-Intel17.0.2	101	9	4	2
inspector2018-Intel18.0.2	104	8	3	1
inspector2018-Intel19.0.0.117	76	39	0	1
inspector2018-Intel19.0.4.227	104	8	4	0
inspector2019-Intel17.0.2	104	10	0	2
inspector2019-Intel18.0.2	105	10	0	1
inspector2019-Intel19.0.0.117	76	37	2	1
inspector2019-Intel19.0.4.227	105	10	1	0
romp-clang8.0.0	99	7	0	10
tsan5.0.2-clang5.0.2	97	6	7	6
tsan6.0.1-clang6.0.1	100	6	5	5
tsan7.1.0-clang7.1.0	97	6	8	5
tsan8.0.1-clang8.0.1	94	9	7	6

Metric report:

Tool-Compiler	successRate	errorRate	precision(min-max)	recall(min-max)	accuracy(min-max)
archer1.0-clang3.9.1	0.8	0.13	1.0-1.0	0.85-0.89	0.92-0.94
archer2.0-clang6.0.0	0.5	0.04	0.52-0.52	0.97-0.97	0.52-0.52
archer2.0-clang6.0.0*	0.9	0.04	0.98-0.98	0.9-0.91	0.94-0.95
inspector2018-Intel17.0.2	0.87	0.02	0.93-0.96	0.85-0.88	0.89-0.92
inspector2018-Intel18.0.2	0.9	0.01	0.94-0.96	0.86-0.9	0.9-0.93
inspector2018-Intel19.0.0.117	0.66	0.01	0.61-0.61	0.95-0.95	0.66-0.66
inspector2018-Intel19.0.4.227	0.9	0.0	0.93-0.95	0.86-0.92	0.9-0.93
inspector2019-Intel17.0.2	0.9	0.02	0.96-0.96	0.86-0.86	0.91-0.91
inspector2019-Intel18.0.2	0.91	0.01	0.96-0.96	0.86-0.86	0.91-0.91
inspector2019-Intel19.0.0.117	0.66	0.01	0.61-0.62	0.95-0.97	0.66-0.68
inspector2019-Intel19.0.4.227	0.91	0.0	0.94-0.96	0.86-0.86	0.91-0.91
romp-clang8.0.0	0.85	0.09	0.96-0.96	0.91-0.91	0.93-0.93
tsan5.0.2-clang5.0.2	0.84	0.05	0.98-0.98	0.79-0.91	0.88-0.95
tsan6.0.1-clang6.0.1	0.86	0.04	0.98-0.98	0.83-0.91	0.9-0.95
tsan7.1.0-clang7.1.0	0.84	0.04	0.96-0.98	0.79-0.91	0.87-0.95
tsan8.0.1-clang8.0.1	0.81	0.05	0.96-0.98	0.76-0.86	0.85-0.92

* Archer 2.0 with "export TSAN_OPTIONS="ignore_noninstrumented_modules=1" * ThreadSanitizer uses LLVM OpenMP runtime with LIBOMP_TSAN_SUPPORT turned on

Metrics formula:

precision (P) = TP/(TP + FP)

recall (R) = TP/(TP +FN)

accuracy (A) = (TP +TN)/(TP +FP +TN +FN)