convert-diff-transformer CLI command / codepath#2197

Draft

djsaunde wants to merge 25 commits intomainfrom diff-transformer

+1,415-52

Commits on Dec 23, 2024

Basic evaluate CLI command / codepath (#2188 )

djsaunde
authored and
Dan Saunders
committed
initial diff attn layer / model conversion implementation (support for llama arch)

djsaunde
authored and
Dan Saunders
committed
Adding script for doing conversion; fixes and updates

djsaunde
authored and
Dan Saunders
committed
adding CLI command for convert-diff-transformer

djsaunde
authored and
Dan Saunders
committed
training fixes, patching, minor cleanup

djsaunde
authored and
Dan Saunders
committed
various improvemnents

djsaunde
authored and
Dan Saunders
committed
various improvemnents

djsaunde
authored and
Dan Saunders
committed
fix model save / load logic

djsaunde
authored and
Dan Saunders
committed
pre-commit fix

djsaunde
authored and
Dan Saunders
committed
moving monkeypatch

djsaunde
authored and
Dan Saunders
committed
differential flash attention 2; cleanup

djsaunde
authored and
Dan Saunders
committed
duplicate code ignore

djsaunde
authored and
Dan Saunders
committed
convert-differential-transformer test coverage

djsaunde
authored and
Dan Saunders
committed
plugin implementation

djsaunde
authored and
Dan Saunders
committed
fixes post-rebase

djsaunde
authored and
Dan Saunders
committed
isolating problematic test

djsaunde
authored and
Dan Saunders
committed
adding split_heads argument for retaining original (Q, K) dimensionanlity

djsaunde
authored and
Dan Saunders
committed
moving tests around for flash_attn install

djsaunde
authored and
Dan Saunders
committed
removing extra pytest xdist args

djsaunde
authored and
Dan Saunders
committed
adding yaml dumper preserving input config format

djsaunde
authored and
Dan Saunders
committed
refactor and fixing test isolation issues

djsaunde
authored and
Dan Saunders
committed
added modeling code; cleanup + refactor
Dan Saunders
committed
fix duplicate-code warnings
Dan Saunders
committed

Commits on Dec 24, 2024

updated custom modeling code
Dan Saunders
committed
progress on modeling code
djsaunde
committed