Code

Add sequence post processor#771

Code

Merged

xenova merged 6 commits intomainhuggingface/transformers.js:mainfrom add-sequence-post-processorhuggingface/transformers.js:add-sequence-post-processorCopy head branch name to clipboardMay 23, 2024

+100-6

Commits

Commits on May 23, 2024

• Add Sequence PostProcessor

  Show description for c1332f8
  xenova

  committed
  c1332f8

• Support return_token_type_ids

  xenova

  committed
  c018f27

• Add llama3 tokenizer to unit tests

  xenova

  committed
  d68afe2

• Add test for allowing user to request for token type ids

  xenova

  committed
  0e5fb41

• Add JSDoc

  xenova

  committed
  3b2f5cd

• Update generate_tests.py

  xenova

  committed
  078672b