Skip to content

Regression metrics

peihunglin edited this page Sep 23, 2019 · 11 revisions

Evaluation platform:

Intel(R) Xeon(R) CPU E5-2695 v4

Summary

full_tool_name Correct (TP/TN) Incorrect (FP/FN) Mix Error
archer3.9.1-clang3.9.1 93 6 2 15
archer6.0.0-clang6.0.0 58 53 0 5
archer6.0.0-clang6.0.0 104 6 1 5
inspector522981-Intel17.0.2 101 9 4 2
inspector522981-Intel18.0.2 104 8 3 1
inspector522981-Intel19.0.0.117 76 39 0 1
inspector522981-Intel19.0.4.227 104 8 4 0
inspector579146-Intel17.0.2 104 10 0 2
inspector579146-Intel18.0.2 105 10 0 1
inspector579146-Intel19.0.0.117 76 37 2 1
inspector579146-Intel19.0.4.227 105 10 1 0
romp-clang8.0.0 99 7 0 10
tsan-clang5.0.2-clang5.0.2 97 6 7 6
tsan-clang6.0.1-clang6.0.1 100 6 5 5
tsan-clang7.1.0-clang7.1.0 97 6 8 5
tsan-clang8.0.1-clang8.0.1 94 9 7 6

Metric report:

full_tool_name successRate errorRate precision(min-max) recall(min-max) accuracy(min-max)
archer3.9.1-clang3.9.1 0.8 0.13 1.0-1.0 0.85-0.89 0.92-0.94
archer6.0.0-clang6.0.0 0.5 0.04 0.52-0.52 0.97-0.97 0.52-0.52
archer6.0.0-clang6.0.0 0.9 0.04 0.98-0.98 0.9-0.91 0.94-0.95
inspector522981-Intel17.0.2 0.87 0.02 0.93-0.96 0.85-0.88 0.89-0.92
inspector522981-Intel18.0.2 0.9 0.01 0.94-0.96 0.86-0.9 0.9-0.93
inspector522981-Intel19.0.0.117 0.66 0.01 0.61-0.61 0.95-0.95 0.66-0.66
inspector522981-Intel19.0.4.227 0.9 0.0 0.93-0.95 0.86-0.92 0.9-0.93
inspector579146-Intel17.0.2 0.9 0.02 0.96-0.96 0.86-0.86 0.91-0.91
inspector579146-Intel18.0.2 0.91 0.01 0.96-0.96 0.86-0.86 0.91-0.91
inspector579146-Intel19.0.0.117 0.66 0.01 0.61-0.62 0.95-0.97 0.66-0.68
inspector579146-Intel19.0.4.227 0.91 0.0 0.94-0.96 0.86-0.86 0.91-0.91
romp-clang8.0.0 0.85 0.09 0.96-0.96 0.91-0.91 0.93-0.93
tsan-clang5.0.2-clang5.0.2 0.84 0.05 0.98-0.98 0.79-0.91 0.88-0.95
tsan-clang6.0.1-clang6.0.1 0.86 0.04 0.98-0.98 0.83-0.91 0.9-0.95
tsan-clang7.1.0-clang7.1.0 0.84 0.04 0.96-0.98 0.79-0.91 0.87-0.95
tsan-clang8.0.1-clang8.0.1 0.81 0.05 0.96-0.98 0.76-0.86 0.85-0.92

Metrics formula:

precision (P) = TP/(TP + FP)

recall (R) = TP/(TP +FN)

accuracy (A) = (TP +TN)/(TP +FP +TN +FN)

Tool & compiler version:

Tool Version Compiler
ThreadSanitizer 8.0.1 Clang 8.0.1
Archer 2.0.0 Clang 6.0.1
Intel Inspector 2019 (build 522981) Intel 19.0.4.227
romp 6a0ad6d Clang 8.0.0