What is the cost for training LADD?

[HoiM]

Thank you for your excellent work!

We are currently trying to apply LADD (Latent Adversarial Diffusion Distillation) on our own 2B text-to-image model.

Can you provide any more experimental details, especially the training cost, e.g. GPU hours, which kind of GPU, global batch size, etc. ?

Appreciate it!

[akasharidas]

We have a working implementation and training code for Latent Adversarial Diffusion Distillation, and achieved good results with multiple base models and resolutions: https://github.com/AMD-AIG-AIMA/AMD-Diffusion-Distillation. In our experiments we found that we can get reasonable results in 1 day on 8x AMD Instinct MI250 GPUs

[HoiM]

Based on our experiments, training with 10k iterations with global batch size being 8 can roughly yield a reasonable result. We trained LADD with 8 A100 (with torch FSDP) and our model is a 2B text-to-image model.