Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not enough benefit due to triplets #17

Open
pradeepiisc opened this issue May 3, 2024 · 0 comments
Open

Not enough benefit due to triplets #17

pradeepiisc opened this issue May 3, 2024 · 0 comments

Comments

@pradeepiisc
Copy link

Hi,
It was a nice article to read about detecting hallucination via generating triplets from the summary and then checking the same against the reference from which the summary was generated. At first the trick looks interesting and useful, but when you actually see the implementation, there's no benefit provided by the approach.

The triplets that are generated are almost like the original sentence itself, specially when it is sent for refchecking to align score or other method, we join the triplets using, say, whitespace or newline, which then resembles the original sentence.

I might as well just call the align score method with the reference and the summary sentence instead of the triplet and get similar results.

Could you provide an argument why using triplets is better than using the original sentence ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant