Skip to content

Commit

Permalink
Update tokenizer error
Browse files Browse the repository at this point in the history
  • Loading branch information
Rocketknight1 committed Feb 19, 2024
1 parent 385abb9 commit 455259e
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion src/transformers/generation/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -921,7 +921,9 @@ def _get_stopping_criteria(
if generation_config.stop_strings is not None:
if "tokenizer" not in kwargs:
raise ValueError(
"To generate with stop strings, you need to pass the model's tokenizer to the `generate` method."
"There are one or more stop strings, either in the arguments to `generate` or in the "
"model's generation config, but we could not locate a tokenizer. When generating with "
"stop strings, you must pass the model's tokenizer to the `tokenizer` argument of `generate`."
)
criteria.append(
StopStringCriteria(stop_strings=generation_config.stop_strings, tokenizer=kwargs["tokenizer"])
Expand Down

0 comments on commit 455259e

Please sign in to comment.