[Question]: Proper way to pass max_grad_norm to ModelTrainer.train

[gdahia]

Question

Greetings! First, thanks for the amazing job with the lib/framework. It is helping me a lot.

Now, I am trying to use max_grad_norm with SGD and I am getting the following error

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/gdahia/Projects/flair-lener/flair_lener/train.py", line 43, in <module>
    train()
  File "/home/gdahia/Projects/flair-lener/flair_lener/train.py", line 31, in train
    trainer.train(
  File "/home/gdahia/.cache/pypoetry/virtualenvs/flair-lener-bVbU5LPC-py3.10/lib/python3.10/site-packages/flair/trainers/trainer.py", line 254, in train
    optimizer_instance = optimizer(self.model.parameters(), **kwargs)
TypeError: SGD.__init__() got an unexpected keyword argument 'max_grad_norm'

Here is the reproducing code snippet

    ...
    tagger = SequenceTagger(
        embeddings=embeddings,
        tag_dictionary=label_dict,
        tag_type=label_type,
        use_rnn=True,
        rnn_type="LSTM",
        hidden_size=100,
        bidirectional=True,
        use_crf=True,
        dropout=0.5,
        word_dropout=0,
        locked_dropout=0,
    )

    trainer = ModelTrainer(tagger, corpus)

    trainer.train(
        "models/ner",
        anneal_factor=0.95,
        patience=20,
        learning_rate=0.015,
        mini_batch_size=10,
        max_epochs=55,
        max_grad_norm=5.0,
    )

and my pyproject.toml:

[tool.poetry.dependencies]
python = "^3.10"
flair = "^0.12.2"
cached-path = "^1.4.0"
torch = "1.13.1"


[tool.poetry.group.dev.dependencies]
ruff = "^0.0.290"
black = "^23.9.1"

I am not sure if I am using it correctly, or if using max_grad_norm should make me use train_custom. If that is a bug, I would also be glad to implement a PR fixing it :D

[helpmefindaname]

Hi @gdahia
the max_grad_norm parameter is not released yet, you can only try it out when installing the master branch directly