Skip to content

Commit

Permalink
change learning rate for multihead ft
Browse files Browse the repository at this point in the history
  • Loading branch information
ilyes319 committed Dec 2, 2024
1 parent c536d11 commit ce09434
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions mace/cli/run_train.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,12 @@ def run(args: argparse.Namespace) -> None:
args.E0s != "average"
), "average atomic energies cannot be used for multiheads finetuning"
# check that the foundation model has a single head, if not, use the first head
args.lr = 0.001
args.ema = True
args.ema_decay = 0.999
logging.info(
"Using multiheads finetuning mode, setting learning rate to 0.001 and EMA to True"
)
if hasattr(model_foundation, "heads"):
if len(model_foundation.heads) > 1:
logging.warning(
Expand Down

0 comments on commit ce09434

Please sign in to comment.