Finetuning Models

[ak2028]

I followed the tutorial at train/finetune_example/mpt-7b-arc-easy--gpu.yaml and added an additional evaluation using icl_tasks: 'eval/yamls/tasks_light.yaml' in order to evaluate accuracy on ARC Easy. As the model finetuned, training loss decreased, but so did accuracy, which appears to be a bug.

I repeated this using the full ARC Easy training set and the same thing occurred. Is there a reason that finetuning causes training loss to decrease but accuracy on evaluation to decrease?

[samhavens]

When you used all of ARC easy, can you share what changes you made to the YAML?

[ak2028]

Sure, I only changed:
data_dir: train/finetune_example/arc-easy/
In arc-easy I have a train.jsonl

I downloaded the data from: https://huggingface.co/datasets/ai2_arc