Skip to content

Regression metrics

peihunglin edited this page Sep 16, 2019 · 11 revisions

Evaluation platform:

Intel(R) Xeon(R) CPU E5-2695 v4

Summary

full_tool_name Correct (TP,TN) Incorrect(FP,FN) Mix(TP/FN,TN/FP) Error
Archer 1.0 93 6 2 15
Archer 2.0 104 6 1 5
Archer 2.0* 58 53 0 5
inspector2019-Intel17.0.2 101 9 4 2
inspector2019-Intel18.0.2 104 8 3 1
inspector2019-Intel19.0.0.117 76 39 0 1
inspector2019-Intel19.0.4.227 104 8 4 0
inspector2018-Intel17.0.2 104 10 0 2
inspector2018-Intel18.0.2 105 10 0 1
inspector2018-Intel19.0.0.117 76 37 2 1
inspector2018-Intel19.0.4.227 105 10 1 0
romp-clang8.0.0 99 7 0 10
tsan-5.0.2 97 6 7 6
tsan-6.0.1 100 6 5 5
tsan-7.1.0 97 6 8 5
tsan-8.0.1 94 9 7 6

Metric report:

tool success rate precision-min precision-max recall-min recall-max accuracy-min accuracy-max
Archer 1.0 0.87 1 1 0.85 0.89 0.92 0.94
Archer 2.0 0.96 0.52 0.52 0.97 0.97 0.52 0.52
Archer 2.0 0.96 0.98 0.98 0.9 0.91 0.94 0.95
inspector2019-Intel17.0.2 0.98 0.93 0.96 0.85 0.88 0.89 0.92
inspector2019-Intel18.0.2 0.99 0.94 0.96 0.86 0.9 0.9 0.93
inspector2019-Intel19.0.0.117 0.99 0.61 0.61 0.95 0.95 0.66 0.66
inspector2019-Intel19.0.4.227 1 0.93 0.95 0.86 0.92 0.9 0.93
inspector2018-Intel17.0.2 0.98 0.96 0.96 0.86 0.86 0.91 0.91
inspector2018-Intel18.0.2 0.99 0.96 0.96 0.86 0.86 0.91 0.91
inspector2018-Intel19.0.0.117 0.99 0.61 0.62 0.95 0.97 0.66 0.68
inspector2018-Intel19.0.4.227 1 0.94 0.96 0.86 0.86 0.91 0.91
romp-clang8.0.0 0.91 0.96 0.96 0.91 0.91 0.93 0.93
tsan-clang5.0.2-clang5.0.2 0.95 0.98 0.98 0.79 0.91 0.88 0.95
tsan-clang6.0.1-clang6.0.1 0.96 0.98 0.98 0.83 0.91 0.9 0.95
tsan-clang7.1.0-clang7.1.0 0.96 0.96 0.98 0.79 0.91 0.87 0.95
tsan-clang8.0.1-clang8.0.1 0.95 0.96 0.98 0.76 0.86 0.85 0.92
draco 0.4 0.58 0.58 0.55 0.55 0.59 0.59

Metrics formula:

precision (P) = TP/(TP + FP)

recall (R) = TP/(TP +FN)

accuracy (A) = (TP +TN)/(TP +FP +TN +FN)

Tool & compiler version:

Tool Version Compiler
ThreadSanitizer 8.0.1 Clang 8.0.1
Archer 2.0.0 Clang 6.0.1
Intel Inspector 2019 (build 522981) Intel 19.0.4.227
romp 6a0ad6d Clang 8.0.0