Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

参数不一致问题: 预训练的reuse_len=256, 下载文件中config.json的reuse_len=null #8

Open
ofooo opened this issue Dec 5, 2019 · 0 comments
Open

参数不一致问题: 预训练的reuse_len=256, 下载文件中config.json的reuse_len=null #8

ofooo opened this issue Dec 5, 2019 · 0 comments

Comments

@ofooo
Copy link

ofooo commented Dec 5, 2019

您好.

我发现有两个参数在训练时和预训练文件中的值不一致.

在https://github.com/brightmart/xlnet_zh的预训练中的1、生成tfrecords:中的配置中: --reuse_len=256 \

在https://github.com/brightmart/xlnet_zh的预训练中的2、训练模型:中的配置中: --mem_len=384 \

在下载预训练文件的config.json中.(12层的小模型, 24层的大模型配置文件都如此)

  "mem_len": null,
  "reuse_len": null,

请问这是什么原因导致的? 这在预测的时候是否会导致xlnet退化成bert?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ofooo