Batch Size and Energy + Force Training

[rschireman]

Hi all, thanks for maintaining this code. I'm attempting to train a model for just ice and I noted that batch sizes should be really small for force training (1-5). I've also confirmed this, but is there a known reason why?

[rschireman]

for example, with the same model parameters, I'll notice a significant difference between batch_size=5 vs batch_size=2, where 2 is the better option

[Linux-cpp-lisp]

Hi @rschireman ,

I'm not aware of a careful study of the reasons for this, but your observation is consistent with everything we've observed. Larger batch sizes seem to work best for very diverse training sets.

[rschireman]

thanks!