Skip to content

Commit

Permalink
Apply suggestions from code review
Browse files Browse the repository at this point in the history
Co-authored-by: Rémy Léone <[email protected]>
  • Loading branch information
Laure-di and remyleone authored Oct 4, 2024
1 parent 022f7e5 commit cd3be0e
Showing 1 changed file with 9 additions and 11 deletions.
20 changes: 9 additions & 11 deletions tutorials/how-to-implement-rag/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ Retrieval-Augmented Generation (RAG) supercharges language models by enabling re
In this comprehensive guide, you'll learn how to implement RAG using LangChain, one of the leading frameworks for developing robust language model applications. We'll combine LangChain with ***Scaleway’s Managed Inference***, ***Scaleway’s PostgreSQL Managed Database*** (featuring pgvector for vector storage), and ***Scaleway’s Object Storage*** for seamless integration and efficient data management.

## Why LangChain?
LangChain simplifies the process of enhancing language models with retrieval capabilities, allowing developers to build scalable, intelligent applications that access external datasets effortlessly. By leveraging LangChain’s modular design and Scaleway’s cloud services, you can unlock the full potential of Retrieval-Augmented Generation.
[LangChain](https://github.com/langchain-ai/langchain) simplifies the process of enhancing language models with retrieval capabilities, allowing developers to build scalable, intelligent applications that access external datasets effortlessly. By leveraging LangChain’s modular design and Scaleway’s cloud services, you can unlock the full potential of Retrieval-Augmented Generation.

## What You’ll Learn
- How to embed text using a sentence transformer using ***Scaleway Manage Inference***
- How to store and query embeddings using ***Scaleway’s Managed PostgreSQL Database*** with pgvector
- How to store and query embeddings using ***Scaleway’s Managed PostgreSQL Database*** with [pgvector](https://github.com/pgvector/pgvector)
- How to manage large datasets efficiently with ***Scaleway Object Storage***

<Macro id="requirements" />
Expand All @@ -39,8 +39,6 @@ Run the following command to install the required packages:

```sh
pip install langchain psycopg2 python-dotenv
```
### Step 2: Create a .env File

Create a .env file and add the following variables. These will store your API keys, database connection details, and other configuration values.

Expand All @@ -49,7 +47,7 @@ Create a .env file and add the following variables. These will store your API ke
# Scaleway API credentials
SCW_ACCESS_KEY=your_scaleway_access_key
SCW_API_KEY=your_scaleway_secret_ke
SCW_API_KEY=your_scaleway_secret_key
# Scaleway managed database (PostgreSQL) credentials
SCW_DB_NAME=your_scaleway_managed_db_name
Expand Down Expand Up @@ -151,13 +149,13 @@ embeddings = OpenAIEmbeddings(

#### What is tiktoken_enabled?

tiktoken is a tokenization library developed by OpenAI, which is optimized for working with GPT-based models (like GPT-3.5 or GPT-4). It transforms text into smaller token units that the model can process.
[`tiktoken`](https://github.com/openai/tiktoken) is a tokenization library developed by OpenAI, which is optimized for working with GPT-based models (like GPT-3.5 or GPT-4). It transforms text into smaller token units that the model can process.

#### Why set tiktoken_enabled=False?

In the context of using Scaleway’s Managed Inference and the `sentence-t5-xxl` model, TikToken tokenization is not necessary because the model you are using (sentence-transformers) works with raw text and handles its own tokenization internally.
Moreover, leaving tiktoken_enabled as True causes issues when sending data to Scaleway’s API because it results in tokenized vectors being sent instead of raw text. Since Scaleway's endpoint expects text and not pre-tokenized data, this mismatch can lead to errors or incorrect behavior.
By setting tiktoken_enabled=False, you ensure that raw text is sent to Scaleway's Managed Inference endpoint, which is what the sentence-transformers model expects to process. This guarantees that the embedding generation process works smoothly with Scaleway's infrastructure.
Moreover, leaving `tiktoken_enabled` as `True` causes issues when sending data to Scaleway’s API because it results in tokenized vectors being sent instead of raw text. Since Scaleway's endpoint expects text and not pre-tokenized data, this mismatch can lead to errors or incorrect behavior.
By setting `tiktoken_enabled=False`, you ensure that raw text is sent to Scaleway's Managed Inference endpoint, which is what the sentence-transformers model expects to process. This guarantees that the embedding generation process works smoothly with Scaleway's infrastructure.
### Step 3: Create a PGVector Store
Expand All @@ -174,7 +172,7 @@ PGVector: This creates the vector store in your PostgreSQL database to store the
## Load and Process Documents
Use the S3FileLoader to load documents and split them into chunks. Then, embed and store them in your PostgreSQL database.
Use the [`S3FileLoader`](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.s3_file.S3FileLoader.html) to load documents and split them into chunks. Then, embed and store them in your PostgreSQL database.
### Step 1: Import Required Modules
Expand Down Expand Up @@ -245,8 +243,8 @@ conn.commit()
- S3FileLoader: The S3FileLoader loads each file individually from your ***Scaleway Object Storage bucket*** using the file's object_key (extracted from the file's metadata). It ensures that only the specific file is loaded from the bucket, minimizing the amount of data being retrieved at any given time.
- RecursiveCharacterTextSplitter: The RecursiveCharacterTextSplitter breaks each document into smaller chunks of text. This is crucial because embeddings models, like those used in Retrieval-Augmented Generation (RAG), typically have a limited context window (the number of tokens they can process at once).
- Chunk Size: Here, the chunk size is set to 480 characters, with an overlap of 20 characters. The choice of 480 characters is based on the context size supported by the embeddings model. Models have a maximum number of tokens they can process in a single pass, often around 512 tokens or fewer, depending on the specific model you are using. To ensure that each chunk fits within this limit, 380 characters provide a buffer, as different models tokenize characters into variable-length tokens.
- Chunk Overlap: The 20-character overlap ensures continuity between chunks, which helps prevent loss of meaning or context between segments.
- `Chunk Size`: Here, the chunk size is set to 480 characters, with an overlap of 20 characters. The choice of 480 characters is based on the context size supported by the embeddings model. Models have a maximum number of tokens they can process in a single pass, often around 512 tokens or fewer, depending on the specific model you are using. To ensure that each chunk fits within this limit, 380 characters provide a buffer, as different models tokenize characters into variable-length tokens.
- `Chunk Overlap`: The 20-character overlap ensures continuity between chunks, which helps prevent loss of meaning or context between segments.
- Embedding the Chunks: For each document, the text is split into smaller chunks using the text splitter, and an embedding is generated for each chunk using the embeddings.embed_query(chunk) function. This function transforms each chunk into a vector representation that can later be used for similarity search.
- Embedding Storage: After generating the embeddings for each chunk, they are stored in a vector database (e.g., PostgreSQL with pgvector) using the vector_store.add_embeddings(embedding, chunk) method. Each embedding is stored alongside its corresponding text chunk, enabling retrieval during a query.
- Avoiding Redundant Processing: The script checks the object_loaded table in PostgreSQL to see if a document has already been processed (i.e., the object_key exists in the table). If it has, the file is skipped, avoiding redundant downloads, vectorization, and database inserts. This ensures that only new or modified documents are processed, reducing the system's computational load and saving both time and resources.
Expand Down

0 comments on commit cd3be0e

Please sign in to comment.