[Question]: Adjusting threshold for NER models

[lunik1]

Question

Is is possible to adjust the threshold for the built in NER models? This was asked previously at #2865, but went unanswered.

[helpmefindaname]

Hi @lunik1
There is no threshold implemented for ner, as the decoding is a bit more complex, so your question can be split into two parts:

• Can I allow NER-Predictions with even lower confidence?
  • No you cannot, as Viterbi is just searching for the most likely sequence. If the tagging of the entity doesn't apear to be most likely, there is no easy way to recreate it.
• Can I restrict NER-Predictions to require a certain confidence?
  • Yes, simply replace the for span in sentence.get_labels("ner") call with [span for span in sentence.get_labels("ner") if span.confidence > threshold] so you only operate on the spans with high enough confidence.

[lunik1]

Thank you for the quick response. Unfortunately I was trying to do the former.