Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: danny-avila/rag_api
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.3.0
Choose a base ref
...
head repository: danny-avila/rag_api
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
  • 10 commits
  • 12 files changed
  • 3 contributors

Commits on Sep 27, 2024

  1. 🚀 refactor: Langchain v0.3 (#74)

    * ↪️refactor: moved packages out of langchain community and fixed embeddings abstraction
    
    * ↪️refactor: update requirements.txt
    
    * ✓chore: removed long list of requirements.txt
    
    * fix: mongo db query and  embed/upload
    
    * refactor: change env var name
    
    * docs: Atlas MongoDB update env var setting
    
    * docs: MongoDB Atlas direction
    
    * docs: update atlas mongoDB directions
    
    * docs: add deprecated env variable
    
    * refactor: mongo db env variable MONGO_VECTOR_COLLECTION backwards compatability add
    FinnConnor authored Sep 27, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    edd8a0c View commit details
  2. 🪨 feat: AWS Bedrock embeddings support (#75)

    * "WIP: adding bedrock embeddings"
    
    * WIP: feat bedrock embeddings support
    
    * feat: aws bedrock embeddings support
    
    * refactor: update aws region var name
    
    * docs: update env variables documentation for bedrock
    
    * docs: add bedrock embeddings provider in list
    FinnConnor authored Sep 27, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    f21b6e7 View commit details
  3. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    fc7b36c View commit details

Commits on Oct 18, 2024

  1. 📄 fix: PDF Image Extract Dependencies (#89)

    FinnConnor authored Oct 18, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    1a7f5fb View commit details

Commits on Oct 22, 2024

  1. 📌chore: security dependency (#92)

    FinnConnor authored Oct 22, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    36ce865 View commit details

Commits on Nov 21, 2024

  1. chore: updating lite requirements to include ollama, openai, and hugg…

    …ingface (#85)
    FinnConnor authored Nov 21, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    aa105bc View commit details

Commits on Dec 4, 2024

  1. 🔒 chore: Update Dependencies for Security & Lite Image (#103)

    * Revert "chore: updating lite requirements to include ollama, openai, and huggingface (#85)"
    
    This reverts commit aa105bc.
    
    * chore: update langchain dependencies to latest versions
    
    * chore: update langchain dependencies to latest versions for requirements.txt
    
    * chore: bump python-multipart for CVE-2024-53981
    danny-avila authored Dec 4, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    e5e9dfa View commit details

Commits on Dec 13, 2024

  1. 🏃‍♂️ chore: make lite docker image lighter (#105)

    stewart-newscorp authored Dec 13, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    662b057 View commit details

Commits on Dec 16, 2024

  1. 🛠️ fix: Enhance Error Logging, Update Dependencies, and Optimize NLTK…

    … Setup (#106)
    
    * chore: remove version specification from docker-compose.yaml
    
    * chore: add better error logging with traceback across the board
    
    * chore: bump unstructured and langchain core packages
    
    * chore: add NLTK data download and disable Unstructured analytics in Dockerfiles
    danny-avila authored Dec 16, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    95a0cd0 View commit details

Commits on Dec 18, 2024

  1. ✨ feat: Add 'entity_id' Parameter (#107)

    * ✨ feat: Add envFile configuration to VSCode launch settings
    
    * 🛠️ fix: Improve debug_mode environment variable handling for consistency
    
    * feat: add entity_id as alternate user_id param
    danny-avila authored Dec 18, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    9e4bb52 View commit details
Showing with 288 additions and 78 deletions.
  1. +2 −1 .vscode/launch.json
  2. +12 −1 Dockerfile
  3. +13 −2 Dockerfile.lite
  4. +10 −3 README.md
  5. +52 −13 config.py
  6. +0 −2 docker-compose.yaml
  7. +145 −26 main.py
  8. +2 −1 models.py
  9. +12 −10 requirements.lite.txt
  10. +14 −10 requirements.txt
  11. +8 −0 store.py
  12. +18 −9 store_factory.py
3 changes: 2 additions & 1 deletion .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -12,7 +12,8 @@
"8000",
"--reload"
],
"jinja": true
"jinja": true,
"envFile": "${workspaceFolder}/.env"
}
]
}
13 changes: 12 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -4,12 +4,23 @@ WORKDIR /app

# Install pandoc and netcat
RUN apt-get update \
&& apt-get install -y pandoc netcat-openbsd \
&& apt-get install -y --no-install-recommends \
pandoc \
netcat-openbsd \
libgl1-mesa-glx \
libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Download standard NLTK data, to prevent unstructured from downloading packages at runtime
RUN python -m nltk.downloader -d /app/nltk_data punkt_tab averaged_perceptron_tagger
ENV NLTK_DATA=/app/nltk_data

# Disable Unstructured analytics
ENV SCARF_NO_ANALYTICS=true

COPY . .

CMD ["python", "main.py"]
15 changes: 13 additions & 2 deletions Dockerfile.lite
Original file line number Diff line number Diff line change
@@ -1,15 +1,26 @@
FROM python:3.10 AS lite
FROM python:3.10-slim AS lite

WORKDIR /app

# Install pandoc and netcat
RUN apt-get update \
&& apt-get install -y pandoc netcat-openbsd \
&& apt-get install -y --no-install-recommends \
pandoc \
netcat-openbsd \
libgl1-mesa-glx \
libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*

COPY requirements.lite.txt .
RUN pip install --no-cache-dir -r requirements.lite.txt

# Download standard NLTK data, to prevent unstructured from downloading packages at runtime
RUN python -m nltk.downloader -d /app/nltk_data punkt_tab averaged_perceptron_tagger
ENV NLTK_DATA=/app/nltk_data

# Disable Unstructured analytics
ENV SCARF_NO_ANALYTICS=true

COPY . .

CMD ["python", "main.py"]
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -61,14 +61,15 @@ The following environment variables are required to run the application:
- `PDF_EXTRACT_IMAGES`: (Optional) A boolean value indicating whether to extract images from PDF files. Default value is "False".
- `DEBUG_RAG_API`: (Optional) Set to "True" to show more verbose logging output in the server console, and to enable postgresql database routes
- `CONSOLE_JSON`: (Optional) Set to "True" to log as json for Cloud Logging aggregations
- `EMBEDDINGS_PROVIDER`: (Optional) either "openai", "azure", "huggingface", "huggingfacetei" or "ollama", where "huggingface" uses sentence_transformers; defaults to "openai"
- `EMBEDDINGS_PROVIDER`: (Optional) either "openai", "bedrock", "azure", "huggingface", "huggingfacetei" or "ollama", where "huggingface" uses sentence_transformers; defaults to "openai"
- `EMBEDDINGS_MODEL`: (Optional) Set a valid embeddings model to use from the configured provider.
- **Defaults**
- openai: "text-embedding-3-small"
- azure: "text-embedding-3-small" (will be used as your Azure Deployment)
- huggingface: "sentence-transformers/all-MiniLM-L6-v2"
- huggingfacetei: "http://huggingfacetei:3000". Hugging Face TEI uses model defined on TEI service launch.
- ollama: "nomic-embed-text"
- bedrock: "amazon.titan-embed-text-v1"
- `RAG_AZURE_OPENAI_API_VERSION`: (Optional) Default is `2023-05-15`. The version of the Azure OpenAI API.
- `RAG_AZURE_OPENAI_API_KEY`: (Optional) The API key for Azure OpenAI service.
- Note: `AZURE_OPENAI_API_KEY` will work but `RAG_AZURE_OPENAI_API_KEY` will override it in order to not conflict with LibreChat setting.
@@ -77,6 +78,11 @@ The following environment variables are required to run the application:
- Note: `AZURE_OPENAI_ENDPOINT` will work but `RAG_AZURE_OPENAI_ENDPOINT` will override it in order to not conflict with LibreChat setting.
- `HF_TOKEN`: (Optional) if needed for `huggingface` option.
- `OLLAMA_BASE_URL`: (Optional) defaults to `http://ollama:11434`.
- `ATLAS_SEARCH_INDEX`: (Optional) the name of the vector search index if using Atlas MongoDB, defaults to `vector_index`
- `MONGO_VECTOR_COLLECTION`: Deprecated for MongoDB, please use `ATLAS_SEARCH_INDEX` and `COLLECTION_NAME`
- `AWS_DEFAULT_REGION`: (Optional) defaults to `us-east-1`
- `AWS_ACCESS_KEY_ID`: (Optional) needed for bedrock embeddings
- `AWS_SECRET_ACCESS_KEY`: (Optional) needed for bedrock embeddings

Make sure to set these environment variables before running the application. You can set them in a `.env` file or as system environment variables.

@@ -87,10 +93,11 @@ Instead of using the default pgvector, we could use [Atlas MongoDB](https://www.
```env
VECTOR_DB_TYPE=atlas-mongo
ATLAS_MONGO_DB_URI=<mongodb+srv://...>
MONGO_VECTOR_COLLECTION=<collection name>
COLLECTION_NAME=<vector collection>
ATLAS_SEARCH_INDEX=<vector search index>
```

The `ATLAS_MONGO_DB_URI` could be the same or different from what is used by LibreChat. Even if it is the same, the `$MONGO_VECTOR_COLLECTION` collection needs to be a completely new one, separate from all collections used by LibreChat. In additional, create a vector search index for `$MONGO_VECTOR_COLLECTION` with the following json:
The `ATLAS_MONGO_DB_URI` could be the same or different from what is used by LibreChat. Even if it is the same, the `$COLLECTION_NAME` collection needs to be a completely new one, separate from all collections used by LibreChat. In addition, create a vector search index for collection above (remember to assign `$ATLAS_SEARCH_INDEX`) with the following json:

```json
{
65 changes: 52 additions & 13 deletions config.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,11 @@
# config.py
import os
import json
import boto3
import logging
from enum import Enum
from datetime import datetime
from dotenv import find_dotenv, load_dotenv
from langchain_community.embeddings import (
HuggingFaceEmbeddings,
HuggingFaceHubEmbeddings,
OllamaEmbeddings,
)
from langchain_openai import AzureOpenAIEmbeddings, OpenAIEmbeddings
from starlette.middleware.base import BaseHTTPMiddleware
from store_factory import get_vector_store

@@ -28,6 +23,7 @@ class EmbeddingsProvider(Enum):
HUGGINGFACE = "huggingface"
HUGGINGFACETEI = "huggingfacetei"
OLLAMA = "ollama"
BEDROCK = "bedrock"


def get_env_variable(
@@ -60,10 +56,10 @@ def get_env_variable(
ATLAS_MONGO_DB_URI = get_env_variable(
"ATLAS_MONGO_DB_URI", "mongodb://127.0.0.1:27018/LibreChat"
)
ATLAS_SEARCH_INDEX = get_env_variable("ATLAS_SEARCH_INDEX", "vector_index")
MONGO_VECTOR_COLLECTION = get_env_variable(
"MONGO_VECTOR_COLLECTION", "vector_collection"
)

"MONGO_VECTOR_COLLECTION", None
) # Deprecated, backwards compatability
CHUNK_SIZE = int(get_env_variable("CHUNK_SIZE", "1500"))
CHUNK_OVERLAP = int(get_env_variable("CHUNK_OVERLAP", "100"))

@@ -80,7 +76,13 @@ def get_env_variable(

logger = logging.getLogger()

debug_mode = get_env_variable("DEBUG_RAG_API", "False").lower() == "true"
debug_mode = os.getenv("DEBUG_RAG_API", "False").lower() in (
"true",
"1",
"yes",
"y",
"t",
)
console_json = get_env_variable("CONSOLE_JSON", "False").lower() == "true"

if debug_mode:
@@ -171,33 +173,58 @@ async def dispatch(self, request, call_next):
).rstrip("/")
HF_TOKEN = get_env_variable("HF_TOKEN", "")
OLLAMA_BASE_URL = get_env_variable("OLLAMA_BASE_URL", "http://ollama:11434")
AWS_ACCESS_KEY_ID = get_env_variable("AWS_ACCESS_KEY_ID", "")
AWS_SECRET_ACCESS_KEY = get_env_variable("AWS_SECRET_ACCESS_KEY", "")

## Embeddings


def init_embeddings(provider, model):
if provider == EmbeddingsProvider.OPENAI:
from langchain_openai import OpenAIEmbeddings

return OpenAIEmbeddings(
model=model,
api_key=RAG_OPENAI_API_KEY,
openai_api_base=RAG_OPENAI_BASEURL,
openai_proxy=RAG_OPENAI_PROXY,
)
elif provider == EmbeddingsProvider.AZURE:
from langchain_openai import AzureOpenAIEmbeddings

return AzureOpenAIEmbeddings(
azure_deployment=model,
api_key=RAG_AZURE_OPENAI_API_KEY,
azure_endpoint=RAG_AZURE_OPENAI_ENDPOINT,
api_version=RAG_AZURE_OPENAI_API_VERSION,
)
elif provider == EmbeddingsProvider.HUGGINGFACE:
from langchain_huggingface import HuggingFaceEmbeddings

return HuggingFaceEmbeddings(
model_name=model, encode_kwargs={"normalize_embeddings": True}
)
elif provider == EmbeddingsProvider.HUGGINGFACETEI:
return HuggingFaceHubEmbeddings(model=model)
from langchain_huggingface import HuggingFaceEndpointEmbeddings

return HuggingFaceEndpointEmbeddings(model=model)
elif provider == EmbeddingsProvider.OLLAMA:
from langchain_ollama import OllamaEmbeddings

return OllamaEmbeddings(model=model, base_url=OLLAMA_BASE_URL)
elif provider == EmbeddingsProvider.BEDROCK:
from langchain_aws import BedrockEmbeddings

session = boto3.Session(
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
region_name=AWS_DEFAULT_REGION,
)
return BedrockEmbeddings(
client=session.client("bedrock-runtime"),
model_id=model,
region_name=AWS_DEFAULT_REGION,
)
else:
raise ValueError(f"Unsupported embeddings provider: {provider}")

@@ -220,6 +247,11 @@ def init_embeddings(provider, model):
)
elif EMBEDDINGS_PROVIDER == EmbeddingsProvider.OLLAMA:
EMBEDDINGS_MODEL = get_env_variable("EMBEDDINGS_MODEL", "nomic-embed-text")
elif EMBEDDINGS_PROVIDER == EmbeddingsProvider.BEDROCK:
EMBEDDINGS_MODEL = get_env_variable(
"EMBEDDINGS_MODEL", "amazon.titan-embed-text-v1"
)
AWS_DEFAULT_REGION = get_env_variable("AWS_DEFAULT_REGION", "us-east-1")
else:
raise ValueError(f"Unsupported embeddings provider: {EMBEDDINGS_PROVIDER}")

@@ -236,12 +268,19 @@ def init_embeddings(provider, model):
mode="async",
)
elif VECTOR_DB_TYPE == VectorDBType.ATLAS_MONGO:
logger.warning("Using Atlas MongoDB as vector store is not fully supported yet.")
# Backward compatability check
if MONGO_VECTOR_COLLECTION:
logger.info(
f"DEPRECATED: Please remove env var MONGO_VECTOR_COLLECTION and instead use COLLECTION_NAME and ATLAS_SEARCH_INDEX. You can set both as same, but not neccessary. See README for more information."
)
ATLAS_SEARCH_INDEX = MONGO_VECTOR_COLLECTION
COLLECTION_NAME = MONGO_VECTOR_COLLECTION
vector_store = get_vector_store(
connection_string=ATLAS_MONGO_DB_URI,
embeddings=embeddings,
collection_name=MONGO_VECTOR_COLLECTION,
collection_name=COLLECTION_NAME,
mode="atlas-mongo",
search_index=ATLAS_SEARCH_INDEX,
)
else:
raise ValueError(f"Unsupported vector store type: {VECTOR_DB_TYPE}")
2 changes: 0 additions & 2 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
version: '3.8'

services:
db:
image: ankane/pgvector:latest
Loading