Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use a pool of OpenAI api keys #15

Open
dcadenas opened this issue Oct 7, 2023 · 1 comment
Open

Use a pool of OpenAI api keys #15

dcadenas opened this issue Oct 7, 2023 · 1 comment

Comments

@dcadenas
Copy link
Contributor

dcadenas commented Oct 7, 2023

Our system will probably have a rate limit bottleneck on the OpenAI endpoint if we happen to be popular. This is partially covered by #14 but we can also prepare a pool of OpenAI keys that we randomly select on function processing.
@dcadenas dcadenas added this to Cleanstr Oct 7, 2023
@dcadenas dcadenas converted this from a draft issue Oct 7, 2023
@dcadenas dcadenas mentioned this issue Oct 7, 2023
Open
@dcadenas dcadenas moved this from 📋 Backlog to 🏗 In progress in Cleanstr Oct 10, 2023
@dcadenas
Copy link
Contributor Author

This is implemented and requires that we add the different api keys in the OPENAI_API_CSV comma separated env variable.

The current implementation has only one key.

Keep in mind that the keys are rate limited per organization.

@dcadenas dcadenas moved this from 🏗 In progress to ✅ Done in Cleanstr Oct 10, 2023
@dcadenas dcadenas removed this from Cleanstr Nov 22, 2023
@dcadenas dcadenas moved this to Done in Verse: Nos Projects Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
Verse: Nos Projects
Archived in project
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dcadenas