Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LiteLLM Integration + Type Fixing + Usage Tracking + Testing #26

Merged
merged 29 commits into from
Nov 6, 2024

Conversation

sidjha1
Copy link
Collaborator

@sidjha1 sidjha1 commented Oct 31, 2024

  • Remove OpenAIModel and just have a LM class that uses LiteLLM
  • Add mypy and pre-commit and get mypy to pass (also add mypy to CI).
  • Add usage tracking with lm.print_total_usage()
  • Add ollama tests with llama3.2:3b
  • Verify that code runs with vLLM
  • Merge in RM and Reranker refactoring from Refactor RM and Reranker #28

Notes:

  • I had to do a bit of random/weird plumbing to get mypy to pass since our types were pretty messed up

@sidjha1 sidjha1 changed the title [WIP] LiteLLM Integration LiteLLM Integration Nov 3, 2024
@sidjha1 sidjha1 changed the title LiteLLM Integration LiteLLM Integration + Type Fixing Nov 3, 2024
@sidjha1 sidjha1 changed the title LiteLLM Integration + Type Fixing LiteLLM Integration + Type Fixing + Usage Tracking Nov 3, 2024
@sidjha1 sidjha1 changed the title LiteLLM Integration + Type Fixing + Usage Tracking LiteLLM Integration + Type Fixing + Usage Tracking + Testing Nov 3, 2024
@sidjha1 sidjha1 requested a review from liana313 November 3, 2024 06:03
lotus/models/lm.py Show resolved Hide resolved
examples/op_examples/agg.py Show resolved Hide resolved
lotus/models/lm.py Show resolved Hide resolved
@sidjha1 sidjha1 merged commit 103e42a into main Nov 6, 2024
5 checks passed
@sidjha1 sidjha1 deleted the sid/litellm branch November 6, 2024 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants