add option to skip special tokens #2118
Annotations
7 warnings
|
The following actions uses node12 which is deprecated and will be forced to run on node16: actions-rs/toolchain@v1, actions/setup-python@v2, actions-rs/cargo@v1. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
|
|
Install Rust
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
|
|
Install Rust
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
|
|
Install Rust
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
|
|
Install Rust
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
|
|
Build:
tokenizers/src/tokenizer/mod.rs#L295
field `encode_special_tokens` is never read
|
|
Build
`tokenizers` (lib) generated 1 warning
|
The logs for this run have expired and are no longer available.
Loading