Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there any way to use T-NER without downloading the model every time? #48

Open
MrDevCop opened this issue Feb 7, 2023 · 2 comments
Open

Comments

@MrDevCop
Copy link

MrDevCop commented Feb 7, 2023

I understand downloading the model once. It gets stored in the cache and is available for the user in that session. But I want to know how to use the method TransformerNER() with a specified model without downloading it every time, even if a session renews. For that to happen I believe I will have to download the model files in the system and somehow point the method TransformerNER to the required paths.
Any help would be much appreciated.

Background: We are trying to use tner library to identify entities in a client's dataset. However, we can access the dataset via a VM on which internet access is not available. But we can transfer files and folders to the VM. So we were thinking of a way to download the models on our own systems and then transfer the required files for TransformerNER method manually to the VM.

PS: We only need to use TransformerNER method.

@asahi417
Copy link
Owner

asahi417 commented Feb 7, 2023

Thanks for using TNER!

We store models on huggingface model hub https://huggingface.co/, and when you load a model on TNER, it should be cached under ~/.cache/huggingface/, and it will use the cache from next time, unless it has been updated. So, I assume you can mount that huggingface cache folder to the VM when you launch it, and TNER should work offline.

@asahi417
Copy link
Owner

asahi417 commented Feb 7, 2023

I use a computing cluster for TNER model training, which also has no internet connection, so the above steps are what I actually do in my environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants