Fix the sequence-parallelism for the dense model architecture#4530

Merged

mrwyattii merged 8 commits intomasterfrom fix-sp-dense

Oct 25, 2023

+42-14

Commits on Oct 17, 2023

fix the sequence-parallelism for the dense models
Reza Yazdani
committed

Commits on Oct 18, 2023

fix the gradient scale for when zero is not enabled
Reza Yazdani
committed
fix comm group for allreduce
tohtana
committed
fix format
tohtana
committed

Commits on Oct 21, 2023

Merge branch 'master' into fix-sp-dense
tjruwase
authored

Commits on Oct 25, 2023