Code

Add support for idefics3 (SmolVLM)#1059

Code

Merged

xenova merged 19 commits intomainhuggingface/transformers.js:mainfrom add-idefics3huggingface/transformers.js:add-idefics3Copy head branch name to clipboardDec 2, 2024

+807-93

Commits

Commits on Nov 29, 2024

• [WIP] Add support for idefics3 (SmolVLM)

  xenova

  committed
  3a02a52

  Copy full SHA for 3a02a52

• Cleanup

  xenova

  committed
  293e378

  Copy full SHA for 293e378

• Update DataTypeMap with 4-bit data types

  xenova

  committed
  ffa43f2

  Copy full SHA for ffa43f2

• Format the model inputs before logging to console

  xenova

  committed
  01dd2d1

  Copy full SHA for 01dd2d1

• Use QUInt8 when quantizing models produced by onnxruntime-genai

  xenova

  committed
  f95475f

  Copy full SHA for f95475f

• auto dtype selection

  xenova

  committed
  3aab729

  Copy full SHA for 3aab729

• Export load_image helper function

  xenova

  committed
  33cbbd7

  Copy full SHA for 33cbbd7

• Add listed support for Idefics3

  xenova

  committed
  832a7ce

  Copy full SHA for 832a7ce

Commits on Dec 2, 2024

• Add support for batched 2d images in idefics3 processor

  xenova

  committed
  f9c59f4

  Copy full SHA for f9c59f4

• Update unit tests

  xenova

  committed
  2e0c5bd

  Copy full SHA for 2e0c5bd

• Add another unit test to ensure correctness of pixel attention mask placement

  xenova

  committed
  cadf689

  Copy full SHA for cadf689

• Move image tokens out of call function

  xenova

  committed
  d3205de

  Copy full SHA for d3205de

• Formatting

  xenova

  committed
  2fd4838

  Copy full SHA for 2fd4838

• Improve auto selection logic

  xenova

  committed
  84c5b7c

  Copy full SHA for 84c5b7c

• Return correct pixel_attention_mask

  xenova

  committed
  45de96a

  Copy full SHA for 45de96a

• Update pixel_attention_mask unit test

  xenova

  committed
  f88500c

  Copy full SHA for f88500c

• Formatting

  xenova

  committed
  b448b4e

  Copy full SHA for b448b4e

• Add idefics3 unit tests

  xenova

  committed
  14a0968

  Copy full SHA for 14a0968

• Increase idefics processor unit test timeout

  xenova

  committed
  e836628

  Copy full SHA for e836628