Code

OLMo1123 classification.#1

Code

Open

vwxyzjn wants to merge 25 commits into2015aroras:shanea/add-olmo11242015aroras/transformers:shanea/add-olmo1124from vwxyzjn:olmo1124_classificationvwxyzjn/transformers:olmo1124_classificationCopy head branch name to clipboard

+2,873

Commits

Commits on Oct 31, 2024

• Add model skeletion with transformers-cli add-new-model-like

  2015aroras

  committed
  8e0ccca

• Convert config to modular, add rms_norm_eps, delete clip_qkv

  2015aroras

  committed
  d6b18e2

• Convert model to modular, add RMSNorm

  2015aroras

  committed
  6605eb9

• Add flash attention with qk norm and no qkv clipping

  2015aroras

  committed
  916a841

• Add decoder layer with RMSNorm after attention/feedforward layers

  2015aroras

  committed
  c19d622

• Add base and causal model

  2015aroras

  committed
  ebec833

• Add converter improvements from OLMo repo

  2015aroras

  committed
  ee2ad8a

• Update weight loading in OLMo to HF converter

  2015aroras

  committed
  84f72ba

• Set correct default for rms_norm_eps

  2015aroras

  committed
  68c6763

• Set correct pipeline_model_mapping in test

  2015aroras

  committed
  8e5a74a

• Run make fixup

  2015aroras

  committed
  a5f92c2

Commits on Nov 4, 2024

• Fix model type

  2015aroras

  committed
  fe2e478

• Merge remote-tracking branch 'upstream/main' into shanea/add-olmo1124

  2015aroras

  committed
  029e843

• Re-run modular conversion

  2015aroras

  committed
  4349938

• Manually set config docs to fix build errors

  2015aroras

  committed
  bd94d9c

• Convert olmo-1124 to olmo_1124 to fix flash attention docs errors

  2015aroras

  committed
  7a0cbbe

• Start updating tests

  2015aroras

  committed
  ada451c

• Update tests

  2015aroras

  committed
  4a2bf2e

• Merge branch 'main' into shanea/add-olmo1124

  2015aroras

  authored
  9a7c61e

Commits on Nov 6, 2024

• Merge branch 'main' into shanea/add-olmo1124

  2015aroras

  authored
  5cd260e

• Copy upstream test_eager_matches_sdpa_inference_1_bfloat16 changes to olmo_1124

  2015aroras

  committed
  a330c90

• Rename input_layernorm and post_attention_layernorm to reflect their ops better

  2015aroras

  committed
  1af9cbd

Commits on Nov 7, 2024

• Use correct tokenizer

  2015aroras

  committed
  083cc93

• Remove test unsupported by GPT2 tokenizer

  2015aroras

  committed
  b7bd6bd

Commits on Nov 12, 2024

• quick push

  vwxyzjn

  committed
  e3a1ede