[Bug]: unable to load upos-multi with SequenceTagger - AttributeError

[saltlas]

Describe the bug

Hi all, I'm trying to load "upos-multi" using the demo code available at https://huggingface.co/flair/upos-multi . I notice this bug actually also happens when you try to use the built-in HuggingFace Inference API on this page. I'm getting the following error:

AttributeError: 'LSTM' object has no attribute '_flat_weights'. Did you mean: '_all_weights'?

I've tried (without success):

• downgrading to transformers 4.30.0
• downgrading to torch 2.0.0

To Reproduce

Run the following code as shown on the huggingface page:

from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/upos-multi")

# make example sentence
sentence = Sentence("Ich liebe Berlin, as they say. ")

# predict POS tags
tagger.predict(sentence)

# print sentence
print(sentence)

# iterate over tokens and print the predicted POS label
print("The following POS tags are found:")
for token in sentence:
    print(token.get_label("upos"))

Expected behavior

Expected code to work, I guess?

Logs and Stack traces

Traceback (most recent call last):
  File "P:\Desktop\spacytest.py", line 31, in <module>
    tagger = SequenceTagger.load("flair/upos-multi")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\hli199\AppData\Local\Programs\Python\Python311\Lib\site-packages\flair\models\sequence_tagger_model.py", line 1036, in load
    return cast("SequenceTagger", super().load(model_path=model_path))
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\hli199\AppData\Local\Programs\Python\Python311\Lib\site-packages\flair\nn\model.py", line 555, in load
    return cast("Classifier", super().load(model_path=model_path))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\hli199\AppData\Local\Programs\Python\Python311\Lib\site-packages\flair\nn\model.py", line 179, in load
    state = load_torch_state(model_file)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\hli199\AppData\Local\Programs\Python\Python311\Lib\site-packages\flair\file_utils.py", line 352, in load_torch_state
    return torch.load(f, map_location="cpu")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\hli199\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\serialization.py", line 1028, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\hli199\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\serialization.py", line 1256, in _legacy_load
    result = unpickler.load()
             ^^^^^^^^^^^^^^^^
  File "C:\Users\hli199\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\rnn.py", line 349, in __setstate__
    for w in self._flat_weights]
             ^^^^^^^^^^^^^^^^^^
  File "C:\Users\hli199\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1695, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'LSTM' object has no attribute '_flat_weights'. Did you mean: '_all_weights'?

Screenshots

No response

Additional Context

No response

Environment

Versions:

Flair

0.13.1

Pytorch

2.1.2+cpu

Transformers

4.36.2

GPU

False

[saltlas]

update: Issue can be fixed by downgrading to torch==1.13.1 so it's an issue with torch.

[GitPeterJ]

It's probably an API change from PyTorch 1.x to 2.x and therefore the model should be reconfigured or retrained. It looks like it has been done for flair/upos-multi since that one does work, are there any plans to also update upos-multi? This model still seems to be used a lot and it would be a shame for it to be a version anchor.

[stefan-it]

Hi @saltlas ,

I re-trained the model with latest Flair release and PyTorch 2.2.1 - it took around 3 days on my GPU.

@alanakbik Am I allowed to prepare PR at Hugging Face Model Hub to update the existing model 🤔

[alanakbik]

@stefan-it that would be great!

[stefan-it]

PR on the Model Hub is here: https://huggingface.co/flair/upos-multi/discussions/3 :)

[KamranMK]

When trying to use this model, I get an error
AttributeError: 'dict' object has no attribute 'embedding_length'. From what I understand it also related to pytorch version difference

[alanakbik]

Could you try again? We just merged @stefan-it 's new version of the model.

[KamranMK]

Thanks guys, it works now :)