Code

[BERT] Add support for sdpa#28802

Code

Merged

amyeroberts merged 22 commits intohuggingface:mainhuggingface/transformers:mainfrom hackyon:sdpa-berthackyon/transformers:sdpa-bertCopy head branch name to clipboardApr 26, 2024

+495-86

Commits

Commits on Mar 7, 2024

• Adding SDPA support for BERT

  hackyon

  committed
  ed6efd0

• Using the proper input name for testing model input in inference()

  hackyon

  committed
  0f34d58

• Adding documentation for SDPA in BERT model page

  hackyon

  committed
  c85acbd

• Use the stable link for the documentation

  hackyon

  committed
  5ce07b3

• Adding a gate to only call .contiguous() for torch < 2.2.0

  hackyon

  committed
  288cc1d

• Additions and fixes to the documentation

  hackyon

  committed
  2afd61f

• Minor updates to documentation

  hackyon

  committed
  fa8b5ad

• Adding extra requirements needed for the contiguous() bug

  hackyon

  committed
  05d5c4e

• Adding "Adapted from" in plcae of the "Copied from"

  hackyon

  committed
  95ec569

• Add benchmark speedup tables to the documentation

  hackyon

  committed
  a07fd89

• Minor fixes to the documentation

  hackyon

  committed
  64334c1

• Use ClapText as a replacemenet for Bert in the Copied-From

  hackyon

  committed
  6a7376d

• Some more fixes for the fix-copies references

  hackyon

  committed
  5ddb6e1

• Overriding the test_eager_matches_sdpa_generate in bert tests to not load with low_cpu_mem_usage

  Show description for 35577eb
  hackyon

  committed
  35577eb

• Undo changes to separate test

  hackyon

  committed
  0e62fe0

• Refactored SDPA self attention code for KV projections

  hackyon

  committed
  5c64480

• Change use_sdpa to attn_implementation

  hackyon

  committed
  9a9bb9b

Commits on Mar 19, 2024

• Merge remote-tracking branch 'upstream/main' into sdpa-bert

  hackyon

  committed
  1a0af20

Commits on Apr 6, 2024

• Merge remote-tracking branch 'upstream/main' into sdpa-bert

  hackyon

  committed
  0965399

Commits on Apr 11, 2024

• Merge remote-tracking branch 'upstream/main' into sdpa-bert

  hackyon

  committed
  b4813a0

Commits on Apr 22, 2024

• Merge remote-tracking branch 'upstream/main' into sdpa-bert

  hackyon

  committed
  e312cd1

• Fix test_sdpa_can_dispatch_on_flash by preparing input (required for MultipleChoice models)

  hackyon

  committed
  66a24c1