From 60418aa2865ef5e3a12b6eddfbfc128a59bc12a4 Mon Sep 17 00:00:00 2001 From: winnie <91998347+gwenwindflower@users.noreply.github.com> Date: Mon, 22 Apr 2024 23:09:44 -0500 Subject: [PATCH] docs(readme): add table of llm api rate limits --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9e56692..506653e 100644 --- a/README.md +++ b/README.md @@ -131,9 +131,11 @@ your_build_dir/ `tbd` has some neat alpha features that infer documentation and tests for your columns. There are multiple supported LLMs via API: Groq running Llama 3 70B, Anthropic Claude 3 Opus, and OpenAI GPT-4 Turbo. They have very different rate limits (these are limitations in the API that `tbd` respects): -- **Groq** 30 requests per minute -- **Claude 3 Opus** 5 requests per minute -- **GPT-4 Turbo** 500 request per minute +| Model | Requests per minute | +|----------------------------|----------------------| +| Groq (Running Llama 3 70B) | 30 | +| Claude 3 Opus | 5 | +| GPT-4 Turbo | 500 | As you can see, if you have anything but a very smol schema, you should stick with OpenAI. When Groq ups their rate limit after they're out of beta, that will be the fastest option, but for now, OpenAI is the best bet. The good news is that GPT-4 Turbo is _really_ good at this task (honestly better than Claude Opus) and pretty dang fast! The results are great in my testing.