Skip to content

Regression metrics

peihunglin edited this page Sep 17, 2019 · 11 revisions

Evaluation platform:

Intel(R) Xeon(R) CPU E5-2695 v4

Summary

full_tool_name Correct (TP,TN) Incorrect(FP,FN) Mix(TP/FN,TN/FP) Error
Archer 1.0 93 6 2 15
Archer 2.0 104 6 1 5
Archer 2.0* 58 53 0 5
inspector2019-Intel17.0.2 101 9 4 2
inspector2019-Intel18.0.2 104 8 3 1
inspector2019-Intel19.0.0.117 76 39 0 1
inspector2019-Intel19.0.4.227 104 8 4 0
inspector2018-Intel17.0.2 104 10 0 2
inspector2018-Intel18.0.2 105 10 0 1
inspector2018-Intel19.0.0.117 76 37 2 1
inspector2018-Intel19.0.4.227 105 10 1 0
romp-clang8.0.0 99 7 0 10
tsan-5.0.2 97 6 7 6
tsan-6.0.1 100 6 5 5
tsan-7.1.0 97 6 8 5
tsan-8.0.1 94 9 7 6

Metric report:

full_tool_name successRate errorRate precision(min~max) recall(min~max) accuracy(min~max)
archer3.9.1-clang3.9.1 0.8 0.13 1.0~1.0 0.85~0.89 0.92~0.94
archer6.0.0-clang6.0.0 0.5 0.04 0.52~0.52 0.97~0.97 0.52~0.52
archer6.0.0-clang6.0.0 0.9 0.04 0.98~0.98 0.9~0.91 0.94~0.95
inspector522981-Intel17.0.2 0.87 0.02 0.93~0.96 0.85~0.88 0.89~0.92
inspector522981-Intel18.0.2 0.9 0.01 0.94~0.96 0.86~0.9 0.9~0.93
inspector522981-Intel19.0.0.117 0.66 0.01 0.61~0.61 0.95~0.95 0.66~0.66
inspector522981-Intel19.0.4.227 0.9 0.0 0.93~0.95 0.86~0.92 0.9~0.93
inspector579146-Intel17.0.2 0.9 0.02 0.96~0.96 0.86~0.86 0.91~0.91
inspector579146-Intel18.0.2 0.91 0.01 0.96~0.96 0.86~0.86 0.91~0.91
inspector579146-Intel19.0.0.117 0.66 0.01 0.61~0.62 0.95~0.97 0.66~0.68
inspector579146-Intel19.0.4.227 0.91 0.0 0.94~0.96 0.86~0.86 0.91~0.91
romp-clang8.0.0 0.85 0.09 0.96~0.96 0.91~0.91 0.93~0.93
tsan-clang5.0.2-clang5.0.2 0.84 0.05 0.98~0.98 0.79~0.91 0.88~0.95
tsan-clang6.0.1-clang6.0.1 0.86 0.04 0.98~0.98 0.83~0.91 0.9~0.95
tsan-clang7.1.0-clang7.1.0 0.84 0.04 0.96~0.98 0.79~0.91 0.87~0.95
tsan-clang8.0.1-clang8.0.1 0.81 0.05 0.96~0.98 0.76~0.86 0.85~0.92

Metrics formula:

precision (P) = TP/(TP + FP)

recall (R) = TP/(TP +FN)

accuracy (A) = (TP +TN)/(TP +FP +TN +FN)

Tool & compiler version:

Tool Version Compiler
ThreadSanitizer 8.0.1 Clang 8.0.1
Archer 2.0.0 Clang 6.0.1
Intel Inspector 2019 (build 522981) Intel 19.0.4.227
romp 6a0ad6d Clang 8.0.0