Replies: 1 comment
-
It looks like a minimal local addition to text-generation-webui/extensions/openai/models.py would bring this to you:
[Edit:] Also to make the http://localhost:5000/docs UI work right, add to text-generation-webui/extensions/openai/typing.py
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hey everyone,
I am working on a client application which interfaces with text-generation-webui's new OpenAI API.
I'm currently generating new text using the completions endpoint. I am wondering if there is any way of retrieving the max context length for the currently loaded model via the API, so I can build my prompt accordingly and avoid going beyond that limit.
I haven't been able to find a method for that, am I missing something? What's the best way to avoid going beyond the model's context length?
Thank you!
Beta Was this translation helpful? Give feedback.
All reactions