Skip to content

Issues: google/sentencepiece

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

37 Open 718 Closed
37 Open 718 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

BPE Dropout tokenizer generates unk at the beginning of sequence
#1071 opened Dec 1, 2024 by AnnaLebedeva
The pip command to install the SentencePiece Python module fails.
#1070 opened Nov 20, 2024 by tprrt
Doesn't seem to work with Python 3.13
#1069 opened Nov 20, 2024 by tprrt
2
Initialized number of seed sentencepieces too low
#1068 opened Nov 14, 2024 by DmitriiP20
Asan detects memory leak in sentencepiece/_sentencepiece.cpython-312-x86_64-linux-gnu.so+0x6f7f4
#1065 opened Nov 4, 2024 by renxida
Compatibility Issue when using v0.2.0 with transformers and tensorflow
#1060 opened Oct 2, 2024 by aws-tianquaw
1
Crashes on out of range inputs depending on other inputs bug
#1051 opened Sep 10, 2024 by colehaus
1
With unigram algorithm, constant piece at end of each sentences does not become a token bug
#1047 opened Aug 29, 2024 by jogardi
Error Attribute Error: type object 'SentencePieceTrainer' has no attribute 'train'. Did you mean: 'Train'?
#1046 opened Aug 23, 2024 by bop578530
1
Zero Width Joiner issue for Sinhala Language
#1031 opened Jun 30, 2024 by Nadil-K
No typings in Python package enhancement
#1030 opened Jun 29, 2024 by marcospgp
1
When I set SPM_PROTOBUF_PROVIDER to "package" in CMakeLists.txt, the compilation fails.
#1029 opened Jun 25, 2024 by hhxdestiny
trainer_interface.cc: Integer value -1 is outside the valid range of values [0, 255] for the enumeration type 'ScriptType' enhancement
#1028 opened Jun 19, 2024 by kcoul
1
Wrong calculation of max_score in unigram_model.cc
#1024 opened Jun 12, 2024 by fairydreaming
How to deal with id
#1023 opened Jun 9, 2024 by 980202006
3
coredump when build with CXXFLAGS -Wp,-D_GLIBCXX_ASSERTIONS
#987 opened Mar 20, 2024 by Henry-ZHR
Allow whitespace-only pieces
#984 opened Feb 26, 2024 by bauwenst
High frequency token segmented into letter sequence when input is a tsv file bug
#967 opened Jan 30, 2024 by TingxunShi
4
Evaluate Profile-Guided Optimization (PGO)
#961 opened Jan 9, 2024 by zamazan4ik
Tips for Termux installation
#950 opened Dec 14, 2023 by Manamama
7
A library that conflicts with the use of protobuf in vcpkg
#927 opened Oct 31, 2023 by hhxdestiny
1
A recent EMNLP work to share about task-adaptive tokenization with variable segmentation
#924 opened Oct 24, 2023 by lsy641
4
Unexpected behavior with sampling of repeated character sequence.
#904 opened Aug 14, 2023 by kellymarchisio
Duplicate tokens in BPE vocabulary
#881 opened Jun 9, 2023 by astanic
1
Patches carried by conda-forge for packaging sentencepiece
#876 opened Jun 6, 2023 by h-vetinari
3
ProTip! Follow long discussions with comments:>50.