section_precision_recall.md

Precision & Recall

Notes:

Precision

Are all results relevant?

$$\text{Precision} = \frac{\text{number of relevant results}}{\text{total number of results}}$$

Notes:

Recall

Are all relevant documents in the results?

$$\text{Recall} = \frac{\text{number of relevant documents that were found}}{\text{total number of relevant documents}}$$

Notes:

Notes: How to evaluate?

Evaluation

• Requires human effort
• Manually annotated corpus:

---

Information Need	iPhone X	Galaxy S10	Cover for Galaxy S10	Battery Pack
smartphone	✓	✓	-	-
apple smartphone	✓	-	-	-
smartphone accessory	-	-	✓	✓

Augment with click-stream logs

Notes:

­

Precision for smartphone? 33%

Recall for smartphone? 50%

Notes:

• Audience question

---

Precision & Recall

• ­ Will never be 100% both, so:
• ­ Rank results according to relevance

Notes:

• How to achieve 100% recall?
• What could be criteria for ranking results?