Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derive the score of entailment #546

Open
elrusa opened this issue Oct 4, 2016 · 1 comment
Open

Derive the score of entailment #546

elrusa opened this issue Oct 4, 2016 · 1 comment

Comments

@elrusa
Copy link

elrusa commented Oct 4, 2016

Hi,

I wonder how it's possible to convert a confidence of either ENTAILMENT or NONENTAILMENT from the result file into just a score of ENTAILMENT, such that

1 - the system is sure that the pair is ENTAILMENT.
0 - the system is sure that the pair is NONENTAILMENT.

As result, I want to use these scores as feature values.

@rzanoli
Copy link
Contributor

rzanoli commented Oct 4, 2016

The confidence is a value in the range is [0,1], and 1 means full confidence. Note that the confidence is about the decision reported in this decision object, and it is not the probability of entailment.

From my poin of view if you want to use these scores as feature values for further classification you might think to have an attribute or feature saying if there is entailment or not (ENTAILMENT, NONENTAILMENT) and a second attribute for the feature weighting (the confidence provided by the classifier). Another possibility would be to use the classifier output as it is (ENTAILMENT, NONENTAILMENT) without considering any confidence score, or you could instead think to apply a filter to increse the precision; e.g., you might decide that if the confidence provided by the classifier is higher then 80% then the ENTAILMENT feature is just the label provided by the classifier(ENTAILMENT, NONENTAILMENT) ; contrary if the confidence is below 80% then the feature label is UNKNOWN.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants