Code

shampoo optim support#1919

Code

Open

winglian wants to merge 9 commits intomainaxolotl-ai-cloud/axolotl:mainfrom shampooaxolotl-ai-cloud/axolotl:shampooCopy head branch name to clipboard

+156-3

Commits

Commits on Sep 18, 2024

• wip shampoo optim support

  winglian

  committed
  eb3eab3

  Copy full SHA for eb3eab3

• additional grafting config types and basic example doc

  winglian

  committed
  52e6249

  Copy full SHA for 52e6249

• fixes

  winglian

  committed
  5334d0f

  Copy full SHA for 5334d0f

• remove accidental duplidcated line

  winglian

  committed
  05f61a0

  Copy full SHA for 05f61a0

• ensure epsilon is cast to float

  winglian

  committed
  84dad0b

  Copy full SHA for 84dad0b

• fix casting of optim args

  winglian

  committed
  69a2938

  Copy full SHA for 69a2938

• ddp shampoo

  winglian

  committed
  beaee36

  Copy full SHA for beaee36

• setup precision config for bf16

  winglian

  committed
  992ea51

  Copy full SHA for 992ea51

Commits on Sep 23, 2024

• shampoo checkpoint save workaround

  winglian

  committed
  17330c0

  Copy full SHA for 17330c0