Skip to content

Commit

Permalink
Merge branch 'main' into remove-reg
Browse files Browse the repository at this point in the history
  • Loading branch information
irenedea authored Sep 18, 2023
2 parents 9569b84 + 2ef85ac commit a3e4b99
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@


class OpenAITokenizerWrapper(AutoTokenizer):

# this API is experimental and for evaluation only. It is subject to change as we add support for training
def __init__(self, name: str) -> None:
try:
import tiktoken
Expand Down

0 comments on commit a3e4b99

Please sign in to comment.