Code

Fully Sharded Data Parallel#489

Code

Merged

camillebrianceau merged 33 commits intoaramis-lab:devaramis-lab/clinicadl:devfrom ncassereau:fsdpncassereau/clinicadl:fsdpCopy head branch name to clipboardApr 5, 2024

+207-52

Commits

Commits on Oct 6, 2023

• fsdp first draft

  ncassereau

  committed
  c33019c

• change how scaler works

  ncassereau

  committed
  12908f8

• Add forgotten change

  ncassereau

  committed
  871d04f

• Merge branch 'dev' into fsdp

  ncassereau

  committed
  21e0ab8

• pep8

  ncassereau

  committed
  03ada6d

• Provide fsdp argument to all DDP instanciation

  ncassereau

  committed
  0f9d096

• working even if fsdp is unavailable

  ncassereau

  committed
  b0f232a

• Linter

  ncassereau

  committed
  91ecc13

• typing for archaic python version

  ncassereau

  committed
  6317e17

• stupid mistake

  ncassereau

  committed
  565127e

• Improved monkeypatching. Now allows source inspection

  ncassereau

  committed
  422bbd2

• Temporarily always activate FSDP to test on github actions (should be reverted later on)

  ncassereau

  committed
  208b7e1

• Temporarily always activate FSDP to test on github actions (should be reverted later on)

  ncassereau

  committed
  4e412db

• Temporarily always activate FSDP to test on github actions (should be reverted later on)

  ncassereau

  committed
  972a988

• Restore standard code and add warning for FSDP before pytorch 2.0

  ncassereau

  committed
  292a5ff

Commits on Oct 10, 2023

• Now using FSDP MixedPrecision instead of autocast

  ncassereau

  committed
  0aa529f

Commits on Nov 30, 2023

• Merge branch 'dev' into fsdp

  ncassereau

  committed
  ef460ad

Commits on Dec 12, 2023

• Merge branch 'dev' into fsdp

  ncassereau

  committed
  f97627e

Commits on Dec 13, 2023

• Solved an issue where the handlers would get duplicated

  ncassereau

  committed
  381f636

• Correct wrong git merge

  ncassereau

  committed
  eea7368

Commits on Dec 14, 2023

• Compute correct validation ?

  ncassereau

  committed
  1fc0a16

• Solve an issue where not all GPU would try a collective communication when the dataset length is not a multiplier of the world_size

  ncassereau

  committed
  8ac8260

• Add eval mode to inference functions

  ncassereau

  committed
  497afe0

• Better warning

  ncassereau

  committed
  0a09928

• Useless import removed

  ncassereau

  committed
  365585f

• FSDP auto wrap

  ncassereau

  committed
  9ab6741

Commits on Dec 15, 2023

• linter

  ncassereau

  committed
  cbf0d2a

Commits on Jan 9, 2024

• fake commit

  ncassereau

  committed
  a1b5207

Commits on Feb 1, 2024

• Merge branch 'dev' into fsdp

  camillebrianceau

  authored
  9edd42e

Commits on Apr 1, 2024

• Last changes

  ncassereau

  committed
  552cfd6

• Merge branch 'dev' into fsdp

  ncassereau

  authored
  35bd1a7

Commits on Apr 2, 2024

• linter

  ncassereau

  committed
  57615d3

Commits on Apr 5, 2024

• Merge branch 'dev' into cb_fsdp

  camillebrianceau

  committed
  793f6f5