New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

预训练语料构造问题 #161

Open

mwei314 opened this issue Apr 27, 2021 · 0 comments

mwei314 commented Apr 27, 2021

albert_zh/create_pretraining_data.py

Line 567 in 652faed

masked_lms.append(MaskedLmInstance(index=index, label=tokens[index]))

中文jieba分词后tokens带有##前缀，而output_tokens去除了##，两者不一致会不会影响预训练效果？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment