Code

Add OLMo November 2024#34551

Code

Merged

ArthurZucker merged 26 commits intohuggingface:mainhuggingface/transformers:mainfrom 2015aroras:shanea/add-olmo11242015aroras/transformers:shanea/add-olmo1124Copy head branch name to clipboardNov 18, 2024

+2,641

Commits

Commits on Nov 15, 2024

• Add model skeletion with transformers-cli add-new-model-like

  2015aroras

  committed
  6e747c2

• Convert config to modular, add rms_norm_eps, delete clip_qkv

  2015aroras

  committed
  a80ffd1

• Convert model to modular, add RMSNorm

  2015aroras

  committed
  ffa794e

• Add flash attention with qk norm and no qkv clipping

  2015aroras

  committed
  75d38f0

• Add decoder layer with RMSNorm after attention/feedforward layers

  2015aroras

  committed
  dbd880d

• Add base and causal model

  2015aroras

  committed
  06c9c44

• Add converter improvements from OLMo repo

  2015aroras

  committed
  b73f6d3

• Update weight loading in OLMo to HF converter

  2015aroras

  committed
  c8d9411

• Set correct default for rms_norm_eps

  2015aroras

  committed
  4e3da14

• Set correct pipeline_model_mapping in test

  2015aroras

  committed
  87d54bb

• Run make fixup

  2015aroras

  committed
  b7939d2

• Fix model type

  2015aroras

  committed
  d39587f

• Re-run modular conversion

  2015aroras

  committed
  30c20f6

• Manually set config docs to fix build errors

  2015aroras

  committed
  cdce157

• Convert olmo-1124 to olmo_1124 to fix flash attention docs errors

  2015aroras

  committed
  3a9c61c

• Start updating tests

  2015aroras

  committed
  949648e

• Update tests

  2015aroras

  committed
  0217f40

• Copy upstream test_eager_matches_sdpa_inference_1_bfloat16 changes to olmo_1124

  2015aroras

  committed
  1bdaa05

• Rename input_layernorm and post_attention_layernorm to reflect their ops better

  2015aroras

  committed
  0b1f2bf

• Use correct tokenizer

  2015aroras

  committed
  9e7c77d

• Remove test unsupported by GPT2 tokenizer

  2015aroras

  committed
  11f67eb

• Create GenerationConfig outside of from_pretrained call

  2015aroras

  committed
  0c2a264

• Use simpler init file structure

  2015aroras

  committed
  a22d936

• Add explicit __all__ to support simplified init

  2015aroras

  committed
  a3cca57

• Make safetensor serialization the default

  2015aroras

  committed
  82a75c2

• Update OLMo November 2024 docs

  2015aroras

  committed
  bfd2e63