Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The difference between Tutorials A "use word embeddings in the pretrained bert model" and Tutorials C "bert+lstm/blstm-crf/enc-dec focus model models" #11

Open
LXM-Emily opened this issue Oct 17, 2022 · 2 comments

Comments

@LXM-Emily
Copy link

Hello, your work is very excellent. However, due to my poor understanding, I have some questions when reading the code, and would like to ask for for your help. That is, BERT is used in Tutorial A to obtain word embedding, while BERT+ BLSTM model in Tutorial C is also used to obtain word embedding. What is the difference between the two? And if I want to use bert to get word embeddings and use blstm as the basic model, should I use tutorial A or tutorial C? I will be very happy if you can reply me.

@gongel
Copy link

gongel commented Oct 17, 2022

Tutorial A is to extract BERT embedding from dataset and has two steps. If you train BERT + LSTM for two tasks, use Tutorial C is better.

@LXM-Emily
Copy link
Author

Thank you for your reply and advice vey much. Can I understand it this way? Tutorial A takes the mean of word vectors for the same words in the dataset, and each word corresponds to a word vector, while Tutorial C takes a sentence as a unit and inputs bert to obtain the word vector of this sentence. Thank you again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants