Skip to content

Commit

Permalink
Update LLM Agents Pipeline README.md (#1352)
Browse files Browse the repository at this point in the history 
Updated agents pipeline README.md

Closes #1307 

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Bhargav Suryadevara (https://github.com/bsuryadevara)

Approvers:
  - Devin Robison (https://github.com/drobison00)

URL: #1352
  • Loading branch information
@bsuryadevara
bsuryadevara authored Nov 13, 2023
1 parent 557dbd6 commit 5d1f815
Show file tree
Hide file tree
Showing 2 changed files with 146 additions and 38 deletions.
18 changes: 0 additions & 18 deletions docs/source/examples/llm/agents/README.md
View file

This file was deleted.

1 change: 1 addition & 0 deletions docs/source/examples/llm/agents/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../../../../../examples/llm/agents/README.md
165 changes: 145 additions & 20 deletions examples/llm/agents/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,45 +15,143 @@ See the License for the specific language governing permissions and
limitations under the License.
-->

# Morpheus LLM Agents Example
# Morpheus LLM Agents Pipeline

<Insert Overview>
## Table of Contents

## Background on Agents
1. [Background Information](#background-information)
- [Purpose](#purpose)
- [LLM Service](#llm-service)
- [Agent type](#agent-type)
- [Agent tools](#agent-tools)
- [LLM Library](#llm-library)
2. [Pipeline Implementation](#pipeline-implementation)
3. [Getting Started](#getting-started)
- [Prerequisites](#prerequisites)
- [Set Environment Variables](#set-environment-variables)
- [Install Dependencies](#install-dependencies)
- [Running the Morpheus Pipeline](#running-the-morpheus-pipeline)
- [Run example (Simple Pipeline)](#run-example-simple-pipeline)
- [Run example (Kafka Pipeline)](#run-example-kafka-pipeline)

<Insert Background on LLM Agents and how they work>
# Background Information

## Simple Example
### Purpose
The Morpheus LLM Agents pipeline is designed to seamlessly integrate Large Language Model (LLM) agents into the Morpheus framework. This implementation focuses on efficiently executing multiple LLM queries using the ReAct agent type, which is tailored for versatile task handling. The use of the Langchain library streamlines the process, minimizing the need for additional system migration.

<Insert Simple Example description>
Within the Morpheus LLM Agents context, these agents act as intermediaries, facilitating communication between users and the LLM service. Their primary role is to execute tools and manage multiple LLM queries, enhancing the LLM's capabilities in solving complex tasks. Agents utilize various tools, such as internet searches, VDB retrievers, calculators, and more, to assist in resolving inquiries, enabling seamless execution of tasks and efficient handling of diverse queries.

### Running the Simple Example
### LLM Service
This pipeline supports various LLM services compatible with our LLMService interface, including OpenAI, NeMo, or local execution using llama-cpp-python. In this example, we'll focus on using OpenAI, chosen for its compatibility with the ReAct agent architecture.

### Agent type
The pipeline supports different agent types, each influencing the pattern for interacting with the LLM. For this example, we'll use the ReAct agent type—a popular and reliable choice.

### Agent tools
Depending on the problem at hand, various tools can be provided to LLM agents, such as internet searches, VDB retrievers, calculators, Wikipedia, etc. In this example, we'll use the internet search tool and an llm-math tool, allowing the LLM agent to perform Google searches and solve math equations.

### LLM Library
The pipeline utilizes the Langchain library to run LLM agents, enabling their execution directly within a Morpheus pipeline. This approach reduces the overhead of migrating existing systems to Morpheus and eliminates the need to replicate work done by popular LLM libraries like llama-index and Haystack.

## Pipeline Implementation
- **InMemorySourceStage**: Manages LLM queries in a DataFrame.
- **KafkaSourceStage**: Consumes LLM queries from the Kafka topic.
- **DeserializationStage**: Converts MessageMeta objects into ControlMessages required by the LLMEngine.
- **LLMEngineStage**: Encompasses the core LLMEngine functionality.
- An `ExtracterNode` extracts the questions from the DataFrame.
- A `LangChainAgentNode` runs the Langchain agent executor for all provided input. This node will utilize the agents
run interface to run the agents asynchronously.
- Finally, the responses are incorporated back into the ControlMessage using a `SimpleTaskHandler`.
- **InMemorySinkStage**: Store the results.

## Getting Started

### Prerequisites

#### Set Environment Variables

Before running the project, ensure that you set the required environment variables. Follow the steps below to obtain and set the API keys for OpenAI and SerpApi.

**OpenAI API Key**

Visit [OpenAI](https://openai.com/) and create an account. Navigate to your account settings to obtain your OpenAI API
key. Copy the key and set it as an environment variable using the following command:

```bash
cd ${MORPHEUS_ROOT}/llm
export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
```

**SerpApi API Key**

Go to [SerpApi](https://serpapi.com/users/sign_up) to register and create an account. Once registered, obtain your
SerpApi API key. Set the API key as an environment variable using the following command:

python main.py agents simple
```bash
export SERPAPI_API_KEY="<YOUR_SERPAPI_API_KEY>"
```

## Kafka Example
#### Install Dependencies

<Insert Kafka Example description>
Install the required dependencies.

### Running the Kafka Example
```bash
mamba env update -n morpheus --file ${MORPHEUS_ROOT}/docker/conda/environments/cuda11.8_examples.yml
```

First, a Kafka cluster must be created in order to run the Kafka example. The cluster is required to allow the persistent pipeline to accept queries for the LLM agents. The Kafka cluster can be created using the following command:
### Running the Morpheus Pipeline

<Insert instructions on starting a kafka cluster>
The top level entrypoint to each of the LLM example pipelines is `examples/llm/main.py`. This script accepts a set
of Options and a Pipeline to run. Baseline options are below, and for the purposes of this document we'll assume a
pipeline option of `agents`:

Once the Kafka cluster is running, the Kafka example can be run using the following command:
### Run example (Simple Pipeline):

This example demonstrates the basic implementation of Morpheus pipeline, showcasing the process of executing LLM queries and managing the generated responses. It uses different stages such as InMemorySourceStage, DeserializationStage, ExtracterNode, LangChainAgentNode, SimpleTaskHandler, and InMemorySinkStage within the pipeline to handle various aspects of query processing and response management.

- Utilizes stages such as InMemorySourceStage and DeserializationStage for consuming and batching LLM queries.
- Incorporates an ExtracterNode for extracting questions and a LangChainAgentNode for executing the Langchain agent executor.
- SimpleTaskHandler to manage the responses generated by the LLMs.
- Stores and manages the results within the pipeline using an InMemorySinkStage.

```bash
cd ${MORPHEUS_ROOT}/llm

python main.py agents kafka
```bash
python exmaples/llm/main.py agents simple [OPTIONS]
```

After the pipeline is running, we need to send messages to the pipeline using the Kafka message bus. In a separate terminal, run the following command:
### Options:
- `--num_threads INTEGER RANGE`
- **Description**: Number of internal pipeline threads to use.
- **Default**: `12`

- `--pipeline_batch_size INTEGER RANGE`
- **Description**: Internal batch size for the pipeline. Can be much larger than the model batch size. Also
used for Kafka consumers.
- **Default**: `1024`

- `--model_max_batch_size INTEGER RANGE`
- **Description**: Max batch size to use for the model.
- **Default**: `64`

- `--model_name TEXT`
- **Description**: The name of the model to use in OpenAI.
- **Default**: `gpt-3.5-turbo-instruct`

- `--repeat_count INTEGER RANGE`
- **Description**: Number of times to repeat the input query. Useful for testing performance.
- **Default**: `1`
- `--help`
- **Description**: Show the help message with options and commands details.


### Run example (Kafka Pipeline):

The Kafka Example in the Morpheus LLM Agents demonstrates an streaming implementation, utilizing Kafka messages to
facilitate the near real-time processing of LLM queries. This example is similar to the Simple example but makes use of
a KafkaSourceStage to stream and retrieve messages from the Kafka topic

First, to run the Kafka example, you need to create a Kafka cluster that enables the persistent pipeline to accept queries for the LLM agents. You can create the Kafka cluster using the following guide: [Quick Launch Kafka Cluster Guide](https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/docs/source/developer_guide/contributing.md#quick-launch-kafka-cluster)

Once the Kafka cluster is running, create Kafka topic to produce input to the pipeline.

```bash
# Set the bootstrap server variable
Expand All @@ -66,7 +164,34 @@ kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --create --topic input
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --alter --topic input --partitions 3
```

Now, we can send messages to the pipeline using the following command:
Now Kafka example can be run using the following command with the below listed options:

```bash
python exmaples/llm/main.py agents kafka [OPTIONS]
```

### Options:
- `--num_threads INTEGER RANGE`
- **Description**: Number of internal pipeline threads to use.
- **Default**: `12`

- `--pipeline_batch_size INTEGER RANGE`
- **Description**: Internal batch size for the pipeline. Can be much larger than the model batch size. Also
used for Kafka consumers.
- **Default**: `1024`

- `--model_max_batch_size INTEGER RANGE`
- **Description**: Max batch size to use for the model.
- **Default**: `64`

- `--model_name TEXT`
- **Description**: The name of the model to use in OpenAI.
- **Default**: `gpt-3.5-turbo-instruct`

- `--help`
- **Description**: Show the help message with options and commands details.

After the pipeline is running, we need to send messages to the pipeline using the Kafka topic. In a separate terminal, run the following command:

```bash
kafka-console-producer.sh --bootstrap-server ${BOOTSTRAP_SERVER} --topic input
Expand Down

0 comments on commit 5d1f815

Please sign in to comment.