add option to skip special tokens · huggingface/tokenizers@f7beeba

Triggered via pull request January 19, 2024 11:16

synchronize #1419

encode-special-tokens

Status Success

Total duration 2m 26s

Artifacts 1

docs-check.yml

on: pull_request

5 warnings

The following actions uses node12 which is deprecated and will be forced to run on node16: actions/setup-python@v1, actions-rs/toolchain@v1, actions/upload-artifact@v2. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/

The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Produced during runtime

Name	Size
documentation Expired	18.7 MB