Skip to content

Commit

Permalink
Update description for max_new_tokens to explain the dynamic setting …
Browse files Browse the repository at this point in the history
…behavior in our docs (#3847)
  • Loading branch information
arnavgarg1 authored Dec 21, 2023
1 parent b34cc10 commit 7e34450
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion ludwig/schema/llms/generation.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@ class LLMGenerationConfig(schema_utils.BaseMarshmallowConfig):
max_new_tokens: Optional[int] = schema_utils.PositiveInteger(
default=32,
allow_none=True,
description="The maximum number of new tokens to generate, ignoring the number of tokens in the input prompt.",
description="The maximum number of new tokens to generate, ignoring the number of tokens in the input prompt. "
"If not set, this is dynamically determined by Ludwig based on either the `max_sequence_length` of the ouput "
"feature, the global_max_sequence_length specified in preprocessing (if specified), or the "
"maximum context length supported by the model (in the order specified).",
parameter_metadata=LLM_METADATA["generation"]["max_new_tokens"],
)

Expand Down

0 comments on commit 7e34450

Please sign in to comment.