Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The outcomes of the two running methods are different. #55

Open
wangChangHai1 opened this issue Mar 20, 2024 · 2 comments
Open

The outcomes of the two running methods are different. #55

wangChangHai1 opened this issue Mar 20, 2024 · 2 comments

Comments

@wangChangHai1
Copy link

Uploading question.docx…

@wangChangHai1
Copy link
Author

Hello, could you explain why the outcomes of these two running methods differ? What kind of segmentation tool was used in the first method?

https://huggingface.co/tner/deberta-v3-large-btc Inference API

from tner import TransformersNER
model = TransformersNER("tner/deberta-v3-large-btc")
model.predict(["Congratulations Local 330 members who were promoted this morning! Steve Kaut to Deputy Chief, Todd Kelly to District Chief, Matt Askea and Brent Scarbrough to Captain, and Will Carter, Jeff Gostkowski, and Dave Goodspeed to Lieutenant. Good luck in your positions. #330forthe330 https://t.co/acHXx6DRY4"])

@asahi417
Copy link
Owner

First of all, the model is deberta large model fine-tuned on BTC dataset https://huggingface.co/datasets/tner/btc, and we fine-tune models on NER task with CRF layer https://towardsdatascience.com/conditional-random-fields-explained-e5b8256da776, which is not available on the inference API at the moment, so when we call the model via inference API, it runs the model with out the CRF layer, which results in the different output from the tner API. In short, CRF layer adds bias of the sequence dependencies (eg. I-PER should come after B-PER) to fix the prediction, so you can expect the prediction from inference API is slightly worse than what you can get from tner API due to the CRF availability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants