【Question】Does DPO trainer support with FSDP?

[yingtongxiong]

Hello, I'm wondering if the DPO trainer supports FSDP and, if so, how I can use it. I look forward to your reply.

[asparius]

trl uses accelerate which supports FSDP. However there is no recommeded config of FSDP in the repo unlike DeepSpeed, so you could refer to this page for FSDP. All in all, DPO and trl supports FSDP but not for online algo like PPO #1726.

[yingtongxiong]

trl uses accelerate which supports FSDP. However there is no recommeded config of FSDP in the repo unlike DeepSpeed, so you could refer to this page for FSDP. All in all, DPO and trl supports FSDP but not for online algo like PPO #1726.

@asparius Thank you very much