Lample and Xuezhe

[cosmozhang]

Hi,
By using the code provided in your file, I cannot replicate the results of Lample's and Xuezhe's study. Is there anything I need to pay attention to specifically?

[LiyuanLucasLiu]

Hi,
Several days ago, we reimplemented Lample's and Xuezhe's study, and our implementations achieve roughly the same performance with their reported score (although slightly worse than Xuezhe). However, we're currently fully occupied by some potential paper submissions and dont have time to clean up and release our codes. I guess I can find some time to release those codes in 20 days.
And for your own implementation, i'm not sure why you cannot replicate...
I would let this issue open, and after i released the code, i would let you know.
Good luck and hope i can help you :-)

[cosmozhang]

Thank you. I mean by the code you guys provided. I did not reimplement them in pytorch. :)
Good luck!

[cosmozhang]

Hi, can I ask what is the intuition behind the function construct_bucket_vb for the labels in util.py?
More specifically, what is this line for? buckets[idx][1].append([label[ind] * label_size + label[ind + 1] for ind in range(0, cur_len)] + [label[cur_len] * label_size + pad_label] + [pad_label * label_size + pad_label] * (thresholds[idx] - cur_len_1))
Thanks!

[LiyuanLucasLiu]

This line is used for label padding, specifically, label padding for CRF (with bucket).
CRF models the transition of states instead of states themselves.
E.g., in Eqs. 1 of our paper, the potential function is calculated based on a (y_{i-1}, y_i) pair.
Here, we try to encode this pair into a single number (label[ind] * label_size + label[ind + 1]).
For other parts of this line, they are just doing padding.

[jiesutd]

Hi @cosmozhang , I have implemented both Lample and Xuezhe Ma's structure using PyTorch PyTorchSeqLabel .

The results are comparable with both of them, all the results are listed and compared in the repository.

The CoNLL03 English NER data is not well generalized, which means there is a big difference between dev and test data. The F1-measure in the test data is not quite stable under different seed number, you may try different seed if the code does not work well.

[cosmozhang]

@jiesutd I also implemented Lample's structure using pytorch last year and got similar results. Thank you.