Skip to content

Commit

Permalink
Obey general token limits
Browse files Browse the repository at this point in the history
- Changed max tokens from 8120 -> 8000
- Works with some models (i.e., Grok) with limits like this
- No effect otherwise, single-response limit only
  • Loading branch information
kofi-bhr authored Oct 23, 2024
1 parent dd4dd2c commit 8324c4b
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion app/lib/.server/llm/constants.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// see https://docs.anthropic.com/en/docs/about-claude/models
export const MAX_TOKENS = 8192;
export const MAX_TOKENS = 8000;

// limits the number of model responses that can be returned in a single request
export const MAX_RESPONSE_SEGMENTS = 2;

0 comments on commit 8324c4b

Please sign in to comment.