Replies: 1 comment
-
I think it's a hyper parameter that'll need tuning based on dataset. For my own use, I just try to do 5-10% of total steps. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello
How should I choose the value for
warmup_steps
and forval_set_size
for the config yaml file? In the example config files 10 warmup steps and a val set size of 0.05 is used but others also used 100 warm up steps and 0.01 or 0.02 for val set size. I have a dataset with around 3800 samples and I'm fine-tuningmeta-llama/Llama-2-13b-chat-hf
.Beta Was this translation helpful? Give feedback.
All reactions