From ffb38f27ad32b5f7c337b50c7c9d0033ca4ac487 Mon Sep 17 00:00:00 2001 From: Ying Hu Date: Fri, 15 Nov 2024 15:45:51 +0800 Subject: [PATCH 1/4] Update AgentQnA README.md for refactor doc structure --- .../docker_compose/intel/cpu/xeon/README.md | 91 ++++++++++++++++++- 1 file changed, 89 insertions(+), 2 deletions(-) diff --git a/AgentQnA/docker_compose/intel/cpu/xeon/README.md b/AgentQnA/docker_compose/intel/cpu/xeon/README.md index 852a0476c..a311ac9b9 100644 --- a/AgentQnA/docker_compose/intel/cpu/xeon/README.md +++ b/AgentQnA/docker_compose/intel/cpu/xeon/README.md @@ -1,3 +1,90 @@ -# Deployment on Xeon +# Single node on-prem deployment with Docker Compose on Xeon Scalable processors +This example showcases a hierarchical multi-agent system for question-answering applications. We deploy the example on Xeon. For LLMs, we use OpenAI models via API calls. For instructions on using open-source LLMs, please refer to the deployment guide [here](../../../../README.md). -We deploy the retrieval tool on Xeon. For LLMs, we support OpenAI models via API calls. For instructions on using open-source LLMs, please refer to the deployment guide [here](../../../../README.md). +## Deployment with docker +1. First, clone this repo. + ``` + export WORKDIR= + cd $WORKDIR + git clone https://github.com/opea-project/GenAIExamples.git + ``` +2. Set up environment for this example
+ ``` + # Example: host_ip="192.168.1.1" or export host_ip="External_Public_IP" + export host_ip=$(hostname -I | awk '{print $1}') + # if you are in a proxy environment, also set the proxy-related environment variables + export http_proxy="Your_HTTP_Proxy" + export https_proxy="Your_HTTPs_Proxy" + # Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1" + export no_proxy="Your_No_Proxy" + + export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/ + #OPANAI_API_KEY if you want to use OpenAI models + export OPENAI_API_KEY= + ``` +3. Deploy the retrieval tool (i.e., DocIndexRetriever mega-service) + + First, launch the mega-service. + ``` + cd $WORKDIR/GenAIExamples/AgentQnA/retrieval_tool + bash launch_retrieval_tool.sh + ``` + Then, ingest data into the vector database. Here we provide an example. You can ingest your own data. + ``` + bash run_ingest_data.sh + ``` +4. Launch Tool service + In this example, we will use some of the mock APIs provided in the Meta CRAG KDD Challenge to demonstrate the benefits of gaining additional context from mock knowledge graphs. + ``` + docker run -d -p=8080:8000 docker.io/aicrowd/kdd-cup-24-crag-mock-api:v0 + ``` +5. Launch `Agent` service + + The configurations of the supervisor agent and the worker agent are defined in the docker-compose yaml file. We currently use openAI GPT-4o-mini as LLM, and llama3.1-70B-instruct (served by TGI-Gaudi) in Gaudi example. To use openai llm, run command below. + + ``` + cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/cpu/xeon + bash launch_agent_service_openai.sh + ``` +6. [Optional] Build `Agent` docker image if pulling images failed. + + ``` + git clone https://github.com/opea-project/GenAIComps.git + cd GenAIComps + docker build -t opea/agent-langchain:latest -f comps/agent/langchain/Dockerfile . + ``` +## Validate services + +First look at logs of the agent docker containers: + +``` +# worker agent +docker logs rag-agent-endpoint +``` + +``` +# supervisor agent +docker logs react-agent-endpoint +``` + +You should see something like "HTTP server setup successful" if the docker containers are started successfully.

+ +Second, validate worker agent: + +``` +curl http://${host_ip}:9095/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{ + "query": "Most recent album by Taylor Swift" + }' +``` + +Third, validate supervisor agent: + +``` +curl http://${host_ip}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{ + "query": "Most recent album by Taylor Swift" + }' +``` + +## How to register your own tools with agent + +You can take a look at the tools yaml and python files in this example. For more details, please refer to the "Provide your own tools" section in the instructions [here](https://github.com/opea-project/GenAIComps/tree/main/comps/agent/langchain/README.md). From 120edffa0506fefd9e0b363cb77c31e48258454d Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 15 Nov 2024 07:47:35 +0000 Subject: [PATCH 2/4] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- .../docker_compose/intel/cpu/xeon/README.md | 30 ++++++++++++------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/AgentQnA/docker_compose/intel/cpu/xeon/README.md b/AgentQnA/docker_compose/intel/cpu/xeon/README.md index a311ac9b9..8d373c2dd 100644 --- a/AgentQnA/docker_compose/intel/cpu/xeon/README.md +++ b/AgentQnA/docker_compose/intel/cpu/xeon/README.md @@ -1,7 +1,9 @@ # Single node on-prem deployment with Docker Compose on Xeon Scalable processors + This example showcases a hierarchical multi-agent system for question-answering applications. We deploy the example on Xeon. For LLMs, we use OpenAI models via API calls. For instructions on using open-source LLMs, please refer to the deployment guide [here](../../../../README.md). ## Deployment with docker + 1. First, clone this repo. ``` export WORKDIR= @@ -9,6 +11,7 @@ This example showcases a hierarchical multi-agent system for question-answering git clone https://github.com/opea-project/GenAIExamples.git ``` 2. Set up environment for this example
+ ``` # Example: host_ip="192.168.1.1" or export host_ip="External_Public_IP" export host_ip=$(hostname -I | awk '{print $1}') @@ -22,17 +25,22 @@ This example showcases a hierarchical multi-agent system for question-answering #OPANAI_API_KEY if you want to use OpenAI models export OPENAI_API_KEY= ``` + 3. Deploy the retrieval tool (i.e., DocIndexRetriever mega-service) First, launch the mega-service. + ``` cd $WORKDIR/GenAIExamples/AgentQnA/retrieval_tool bash launch_retrieval_tool.sh ``` + Then, ingest data into the vector database. Here we provide an example. You can ingest your own data. + ``` bash run_ingest_data.sh ``` + 4. Launch Tool service In this example, we will use some of the mock APIs provided in the Meta CRAG KDD Challenge to demonstrate the benefits of gaining additional context from mock knowledge graphs. ``` @@ -40,19 +48,21 @@ This example showcases a hierarchical multi-agent system for question-answering ``` 5. Launch `Agent` service - The configurations of the supervisor agent and the worker agent are defined in the docker-compose yaml file. We currently use openAI GPT-4o-mini as LLM, and llama3.1-70B-instruct (served by TGI-Gaudi) in Gaudi example. To use openai llm, run command below. + The configurations of the supervisor agent and the worker agent are defined in the docker-compose yaml file. We currently use openAI GPT-4o-mini as LLM, and llama3.1-70B-instruct (served by TGI-Gaudi) in Gaudi example. To use openai llm, run command below. + + ``` + cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/cpu/xeon + bash launch_agent_service_openai.sh + ``` - ``` - cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/cpu/xeon - bash launch_agent_service_openai.sh - ``` 6. [Optional] Build `Agent` docker image if pulling images failed. - ``` - git clone https://github.com/opea-project/GenAIComps.git - cd GenAIComps - docker build -t opea/agent-langchain:latest -f comps/agent/langchain/Dockerfile . - ``` + ``` + git clone https://github.com/opea-project/GenAIComps.git + cd GenAIComps + docker build -t opea/agent-langchain:latest -f comps/agent/langchain/Dockerfile . + ``` + ## Validate services First look at logs of the agent docker containers: From 6fe070819f2a5dd4f58126feeb847a78bb66d893 Mon Sep 17 00:00:00 2001 From: Ying Hu Date: Fri, 15 Nov 2024 16:12:44 +0800 Subject: [PATCH 3/4] Create AgentQnA README.md for refactor doc for gaudi deploy --- .../docker_compose/intel/hpu/gaudi/README.md | 103 ++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 AgentQnA/docker_compose/intel/hpu/gaudi/README.md diff --git a/AgentQnA/docker_compose/intel/hpu/gaudi/README.md b/AgentQnA/docker_compose/intel/hpu/gaudi/README.md new file mode 100644 index 000000000..7cf176f80 --- /dev/null +++ b/AgentQnA/docker_compose/intel/hpu/gaudi/README.md @@ -0,0 +1,103 @@ +# Single node on-prem deployment AgentQnA on Gaudi + +This example showcases a hierarchical multi-agent system for question-answering applications. We deploy the example on Gaudi using open-source LLMs, +For more details, please refer to the deployment guide [here](../../../../README.md). + +## Deployment with docker + +1. First, clone this repo. + ``` + export WORKDIR= + cd $WORKDIR + git clone https://github.com/opea-project/GenAIExamples.git + ``` +2. Set up environment for this example
+ + ``` + # Example: host_ip="192.168.1.1" or export host_ip="External_Public_IP" + export host_ip=$(hostname -I | awk '{print $1}') + # if you are in a proxy environment, also set the proxy-related environment variables + export http_proxy="Your_HTTP_Proxy" + export https_proxy="Your_HTTPs_Proxy" + # Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1" + export no_proxy="Your_No_Proxy" + + export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/ + # for using open-source llms + export HUGGINGFACEHUB_API_TOKEN= + # Example export HF_CACHE_DIR=$WORKDIR so that no need to redownload every time + export HF_CACHE_DIR= + + ``` + +3. Deploy the retrieval tool (i.e., DocIndexRetriever mega-service) + + First, launch the mega-service. + + ``` + cd $WORKDIR/GenAIExamples/AgentQnA/retrieval_tool + bash launch_retrieval_tool.sh + ``` + + Then, ingest data into the vector database. Here we provide an example. You can ingest your own data. + + ``` + bash run_ingest_data.sh + ``` + +4. Launch Tool service + In this example, we will use some of the mock APIs provided in the Meta CRAG KDD Challenge to demonstrate the benefits of gaining additional context from mock knowledge graphs. + ``` + docker run -d -p=8080:8000 docker.io/aicrowd/kdd-cup-24-crag-mock-api:v0 + ``` +5. Launch `Agent` service + + To use open-source LLMs on Gaudi2, run commands below. + ``` + cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/hpu/gaudi + bash launch_tgi_gaudi.sh + bash launch_agent_service_tgi_gaudi.sh + ``` +6. [Optional] Build `Agent` docker image if pulling images failed. + + ``` + git clone https://github.com/opea-project/GenAIComps.git + cd GenAIComps + docker build -t opea/agent-langchain:latest -f comps/agent/langchain/Dockerfile . + ``` + +## Validate services + +First look at logs of the agent docker containers: + +``` +# worker agent +docker logs rag-agent-endpoint +``` + +``` +# supervisor agent +docker logs react-agent-endpoint +``` + +You should see something like "HTTP server setup successful" if the docker containers are started successfully.

+ +Second, validate worker agent: + +``` +curl http://${host_ip}:9095/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{ + "query": "Most recent album by Taylor Swift" + }' +``` + +Third, validate supervisor agent: + +``` +curl http://${host_ip}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{ + "query": "Most recent album by Taylor Swift" + }' +``` + +## How to register your own tools with agent + +You can take a look at the tools yaml and python files in this example. For more details, please refer to the "Provide your own tools" section in the instructions [here](https://github.com/opea-project/GenAIComps/tree/main/comps/agent/langchain/README.md). From ddb6134a45067f1f15a6f96f7cc39bb28a41627b Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 15 Nov 2024 08:13:03 +0000 Subject: [PATCH 4/4] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- AgentQnA/docker_compose/intel/hpu/gaudi/README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/AgentQnA/docker_compose/intel/hpu/gaudi/README.md b/AgentQnA/docker_compose/intel/hpu/gaudi/README.md index 7cf176f80..21735e398 100644 --- a/AgentQnA/docker_compose/intel/hpu/gaudi/README.md +++ b/AgentQnA/docker_compose/intel/hpu/gaudi/README.md @@ -1,6 +1,6 @@ # Single node on-prem deployment AgentQnA on Gaudi -This example showcases a hierarchical multi-agent system for question-answering applications. We deploy the example on Gaudi using open-source LLMs, +This example showcases a hierarchical multi-agent system for question-answering applications. We deploy the example on Gaudi using open-source LLMs, For more details, please refer to the deployment guide [here](../../../../README.md). ## Deployment with docker @@ -26,8 +26,8 @@ For more details, please refer to the deployment guide [here](../../../../README # for using open-source llms export HUGGINGFACEHUB_API_TOKEN= # Example export HF_CACHE_DIR=$WORKDIR so that no need to redownload every time - export HF_CACHE_DIR= - + export HF_CACHE_DIR= + ``` 3. Deploy the retrieval tool (i.e., DocIndexRetriever mega-service) @@ -53,11 +53,13 @@ For more details, please refer to the deployment guide [here](../../../../README 5. Launch `Agent` service To use open-source LLMs on Gaudi2, run commands below. + ``` cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/hpu/gaudi bash launch_tgi_gaudi.sh bash launch_agent_service_tgi_gaudi.sh ``` + 6. [Optional] Build `Agent` docker image if pulling images failed. ```