feat(llm): add integration for Mistral, Gemini and Claude #169
Conversation
There was a problem hiding this comment.
Thanks for the PR @sarthakkapila 🙏
Quite ambitious and kind of you to jump into this integration! Two high-level comments:
- I think it would be easier to split this and have 1PR / integration
- Not sure whether you tested the code or used a coding assistant, but I've spotted a few parts in the comments that, I think, cannot run. Would you mind fixing those?
Let me know what you think!
| CLAUDE_API_KEY= | ||
| CLAUDE_MODEL='claude-3-opus-20240229' | ||
| GEMINI_API_KEY= | ||
| GEMINI_MODEL='GeminiPro_1.5_Pro' |
There was a problem hiding this comment.
isn't it "gemini-1.5-pro-latest"? Where did you find this model reference?
| anthropic = "^0.26.1" | ||
| google-generativeai = "^0.5.4" | ||
| mistralai = "^0.2.0" | ||
| pytest = "^8.2.1" |
There was a problem hiding this comment.
pytest is only needed for the tests, and it's already included in the "test" group, would you mind removing it from here please? :)
| google-generativeai = "^0.5.4" | ||
| mistralai = "^0.2.0" | ||
| pytest = "^8.2.1" | ||
| mistral = "^18.0.1" |
There was a problem hiding this comment.
Why the dupplicated dep? we only need https://github.com/mistralai/client-python right?
|
|
||
| MISTRAL_MODEL: str = os.environ.get("MISTRAL_MODEL", "open-mixtral-8x7b") | ||
| CLAUDE_MODEL: str = os.environ.get("CLAUDE_MODEL", "claude-3-opus-20240229") | ||
| GEMINI_MODEL: str = os.environ.get("GEMINI_MODEL", "GeminiPro_1.5_Pro") |
There was a problem hiding this comment.
same here, I can't find that model reference in the API documentation
| Opus: str = "claude-3-opus-20240229" | ||
| Sonnet: str = "claude-3-sonnet-20240229" | ||
| Haiku: str = "claude-3-haiku-20240307" |
There was a problem hiding this comment.
For the sake of conventions, could you use UPPER_CASE for the class attributes of enums please?
| stream = cast( | ||
| self._client.messages.create( | ||
| messages=[ | ||
| {"role": "system", "content": _system}], | ||
| model=self.model, | ||
| temperature=self.temperature, | ||
| max_tokens=2048, | ||
| stream=True, | ||
| # top_p=1, | ||
| # stop=None, | ||
| # stream=True, | ||
| # stream_options={"include_usage": True}, | ||
| ) | ||
| ) |
There was a problem hiding this comment.
I might be missing something but this cannot work: cast requires two positional arguments and the first one being the output type the object will be casted to
| GeminiPro_1_5_Pro: str = "gemini-1.5-pro-latest" | ||
| Gemini_1_5_Flash: str = "gemini-1.5-flash-latest" | ||
| Gemini_1_0_Pro: str = "gemini-1.0-pro" |
| # Prepare the request | ||
| _system = CHAT_PROMPT if not system else f"{CHAT_PROMPT} {system}" | ||
|
|
||
| stream = cast( |
There was a problem hiding this comment.
same here, I don't think that can work 🤔
| @@ -0,0 +1 @@ | |||
| OpenAI | |||
| from groq import AuthenticationError as GAuthError | ||
| from groq import NotFoundError as GNotFoundError |
There was a problem hiding this comment.
why remove that import? this breaks the groq tests
frgfm
changed the title
p.s. thank you so much 🙏 for pointing the mistakes in detail as this helped me a lot. |
Closes #129
Hey! first time contributing to this repo.
Added support for mistral, claude and gemini.
Not really sure about the test file, might need some help.
Thanks