How to continue a previously terminated training process?

[kuntalg97]

Hi,

Is there a way to continue a previously terminated training process? I'm trying to train an Allegro model using nequip-train and my run got terminated due to some cluster issues before any of the stopping criteria were met. My results directory has been generated with the log file as well. How do I continue this run, from the point where the training stopped?

Thanks!
Kuntal

[cw-tan]

Hi @kuntalg97,

You should be able to restart training by just calling the same nequip-train config.yaml command you used to start training the first time. It might also be helpful to know which version of nequip you're using. The restart logic for the latest release can be found here, but tldr is that you should be able to just use the same command you used for the fresh start. Let us know how it goes!

[kuntalg97]

Thanks! It worked!

[Linux-cpp-lisp]

Closing as duplicate of mir-group/nequip#455 (reply in thread).