remove multiprocessing and enhance ut for coverage.

opea-project · Dec 12, 2024 · d35617e · d35617e
1 parent d1266b1
commit d35617e
Show file tree

Hide file tree

Showing 47 changed files with 2,402 additions and 102 deletions.
diff --git a/comps/agent/langchain/README.md b/comps/agent/langchain/README.md
@@ -11,10 +11,11 @@ We currently support the following types of agents:
 1. ReAct: use `react_langchain` or `react_langgraph` or `react_llama` as strategy. First introduced in this seminal [paper](https://arxiv.org/abs/2210.03629). The ReAct agent engages in "reason-act-observe" cycles to solve problems. Please refer to this [doc](https://python.langchain.com/v0.2/docs/how_to/migrate_agent/) to understand the differences between the langchain and langgraph versions of react agents. See table below to understand the validated LLMs for each react strategy.
 2. RAG agent: use `rag_agent` or `rag_agent_llama` strategy. This agent is specifically designed for improving RAG performance. It has the capability to rephrase query, check relevancy of retrieved context, and iterate if context is not relevant. See table below to understand the validated LLMs for each rag agent strategy.
 3. Plan and execute: `plan_execute` strategy. This type of agent first makes a step-by-step plan given a user request, and then execute the plan sequentially (or in parallel, to be implemented in future). If the execution results can solve the problem, then the agent will output an answer; otherwise, it will replan and execute again.
+4. SQL agent: use `sql_agent_llama` or `sql_agent` strategy. This agent is specifically designed and optimized for answering questions aabout data in SQL databases. For more technical details read descriptions [here](src/strategy/sqlagent/README.md).
 
 **Note**:
 
-1. Due to the limitations in support for tool calling by TGI and vllm, we have developed subcategories of agent strategies (`rag_agent_llama` and `react_llama`) specifically designed for open-source LLMs served with TGI and vllm.
+1. Due to the limitations in support for tool calling by TGI and vllm, we have developed subcategories of agent strategies (`rag_agent_llama`, `react_llama` and `sql_agent_llama`) specifically designed for open-source LLMs served with TGI and vllm.
 2. For advanced developers who want to implement their own agent strategies, please refer to [Section 5](#5-customize-agent-strategy) below.
 
 ### 1.2 LLM engine
@@ -25,14 +26,16 @@ Agents use LLM for reasoning and planning. We support 3 options of LLM engine:
 2. Open-source LLMs served with vllm. Follow the instructions in [Section 2.2.2](#222-start-agent-microservices-with-vllm).
 3. OpenAI LLMs via API calls. To use OpenAI llms, specify `llm_engine=openai` and `export OPENAI_API_KEY=<your-openai-key>`
 
-| Agent type       | `strategy` arg    | Validated LLMs (serving SW)                                                                                                                                                                                    | Notes                                                                                                                           |
-| ---------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
-| ReAct            | `react_langchain` | [llama3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) (tgi-gaudi)                                                                                                                  | Only allows tools with one input variable                                                                                       |
-| ReAct            | `react_langgraph` | GPT-4o-mini, [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) (vllm-gaudi),                                                                                               | if using vllm, need to specify `--enable-auto-tool-choice --tool-call-parser ${model_parser}`, refer to vllm docs for more info |
-| ReAct            | `react_llama`     | [llama3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) (tgi-gaudi)                                                                                                                  | Recommended for open-source LLMs                                                                                                |
-| RAG agent        | `rag_agent`       | GPT-4o-mini                                                                                                                                                                                                    |                                                                                                                                 |
-| RAG agent        | `rag_agent_llama` | [llama3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) (tgi-gaudi)                                                                                                                  | Recommended for open-source LLMs, only allows 1 tool with input variable to be "query"                                          |
-| Plan and execute | `plan_execute`    | GPT-4o-mini, [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) (vllm-gaudi), [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) (vllm-gaudi) |                                                                                                                                 |
+| Agent type       | `strategy` arg    | Validated LLMs (serving SW)                                                                                                                                                                                    | Notes                                                                                                                                                                                                                                                                                                                                           |
+| ---------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| ReAct            | `react_langchain` | [llama3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) (tgi-gaudi)                                                                                                                  | Only allows tools with one input variable                                                                                                                                                                                                                                                                                                       |
+| ReAct            | `react_langgraph` | GPT-4o-mini, [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) (vllm-gaudi),                                                                                               | if using vllm, need to specify `--enable-auto-tool-choice --tool-call-parser ${model_parser}`, refer to vllm docs for more info                                                                                                                                                                                                                 |
+| ReAct            | `react_llama`     | [llama3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) (tgi-gaudi) (vllm-gaudi)                                                                                                     | Recommended for open-source LLMs                                                                                                                                                                                                                                                                                                                |
+| RAG agent        | `rag_agent`       | GPT-4o-mini                                                                                                                                                                                                    |                                                                                                                                                                                                                                                                                                                                                 |
+| RAG agent        | `rag_agent_llama` | [llama3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) (tgi-gaudi) (vllm-gaudi)                                                                                                     | Recommended for open-source LLMs, only allows 1 tool with input variable to be "query"                                                                                                                                                                                                                                                          |
+| Plan and execute | `plan_execute`    | GPT-4o-mini, [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) (vllm-gaudi), [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) (vllm-gaudi) | Currently, due to some issues with guaided decoding of vllm-gaudi, this strategy does not work properly with vllm-gaudi. We are actively debugging. Stay tuned. In the meanwhile, you can use OpenAI's models with this strategy.                                                                                                               |
+| SQL agent        | `sql_agent_llama` | [llama3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) (vllm-gaudi)                                                                                                                 | database query tool is natively integrated using Langchain's [QuerySQLDataBaseTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.sql_database.tool.QuerySQLDataBaseTool.html#langchain_community.tools.sql_database.tool.QuerySQLDataBaseTool). User can also register their own tools with this agent. |
+| SQL agent        | `sql_agent`       | GPT-4o-mini                                                                                                                                                                                                    | database query tool is natively integrated using Langchain's [QuerySQLDataBaseTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.sql_database.tool.QuerySQLDataBaseTool.html#langchain_community.tools.sql_database.tool.QuerySQLDataBaseTool). User can also register their own tools with this agent. |
 
 ### 1.3 Tools
 
@@ -120,12 +123,12 @@ Once microservice starts, user can use below script to invoke.
 
 ```bash
 curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
-     "query": "What is the weather today in Austin?"
+     "query": "What is OPEA project?"
     }'
 
 # expected output
 
-data: 'The temperature in Austin today is 78°F.</s>'
+data: 'The OPEA project is .....</s>' # just showing partial example here.
 
 data: [DONE]
 
@@ -210,4 +213,4 @@ data: [DONE]
 ## 5. Customize agent strategy
 
 For advanced developers who want to implement their own agent strategies, you can add a separate folder in `src\strategy`, implement your agent by inherit the `BaseAgent` class, and add your strategy into the `src\agent.py`. The architecture of this agent microservice is shown in the diagram below as a reference.
-![Architecture Overview](agent_arch.jpg)
+![Architecture Overview](assets/agent_arch.jpg)
diff --git a/comps/agent/langchain/agent_arch.jpg → comps/agent/langchain/assets/agent_arch.jpg b/comps/agent/langchain/agent_arch.jpg → comps/agent/langchain/assets/agent_arch.jpg
diff --git a/comps/agent/langchain/assets/sql_agent.png b/comps/agent/langchain/assets/sql_agent.png
diff --git a/comps/agent/langchain/assets/sql_agent_llama.png b/comps/agent/langchain/assets/sql_agent_llama.png
diff --git a/comps/agent/langchain/log b/comps/agent/langchain/log
@@ -0,0 +1,46 @@
+[HumanMessage(content="what's the most recent album from the founder of ysl records?", id='cfde4aba-0464-4ad9-bd1c-d3fc40bbb46e'), AIMessage(content='', addi
+tional_kwargs={'tool_calls': [ChatCompletionOutputToolCall(function=ChatCompletionOutputFunctionDefinition(arguments={'query': 'founder of YSL Records'}, name
+e='duckduckgo_search', description=None), id='142bd66e-7ec6-4381-bcb6-2d0bd4fcecd3', type='function')]}, id='6430421d-2238-452b-9fbe-4d6cc8bc0cd8', tool_call
+s=[{'name': 'duckduckgo_search', 'args': {'query': 'founder of YSL Records'}, 'id': '142bd66e-7ec6-4381-bcb6-2d0bd4fcecd3', 'type': 'tool_call'}]), ToolMessa
+ge(content='Prosecutors allege the chart-topping artist\'s label Young Stoner Life Records also stands for Young Slime ... a YSL co-founder who reached a ple
+a deal in 2022 and claimed Young Thug was a part ... In 2016, Young Thug founded YSL Records, with its full name, "Young Stoner Life," perfectly representing
+ the enigmatic persona of the label. YSL Records quickly became known for its ... The longest criminal trial in the State of Georgia\'s history has been plag
+ued by endless problems, including the recusal of not one but two judges in charge of the case. Young Thug and his lawyer ... Prosecutors say YSL - the acron
+ym for the artist\'s label, Young Stoner Life Records - also stands for Young Slime Life, an Atlanta-based street gang affiliated with the national Bloods ga
+ng. YSL Records calls its roster of artists the "Slime Family." ... YSL co-founder Walter Murphy entered a guilty plea on a single count of conspiracy to vio
+late the state\'s Racketeer Influenced ...', name='duckduckgo_search', id='1d9ebf6b-bf62-469a-8d45-365c0fa15bba', tool_call_id='142bd66e-7ec6-4381-bcb6-2d0bd
+4fcecd3'), HumanMessage(content='Retrieved document is not sufficient or relevant to answer the query. Reformulate the query to search knowledge base again.'
+, id='6f2fb9db-e5b4-4483-b69a-06122c996eb9'), AIMessage(content='', additional_kwargs={'tool_calls': [ChatCompletionOutputToolCall(function=ChatCompletionOut
+putFunctionDefinition(arguments={'query': 'YSL Records founder'}, name='duckduckgo_search', description=None), id='6d073578-5bf1-449e-8d32-4f3b5f999a06', typ
+e='function')]}, id='50a644af-3117-4513-be56-5fb4b16d0481', tool_calls=[{'name': 'duckduckgo_search', 'args': {'query': 'YSL Records founder'}, 'id': '6d0735
+78-5bf1-449e-8d32-4f3b5f999a06', 'type': 'tool_call'}]), ToolMessage(content='That same year, he founded the YSL record label, which the rapper has used to p
+ropel close friends and family members to industry success. The Birth Of YSL Records And Its Impact On The Music Industry In 2016, Young Thug founded YSL Rec
+ords, with its full name, "Young Stoner Life," perfectly representing the enigmatic persona of ... Young Thug, who runs the Young Stoner Life label, has been
+ accused of co-founding the Young Slime Life Atlanta gang and violating the RICO act, among other charges. Here\'s what to know about the ... The rapper foun
+dead the record label Young Stoner Life in 2016 as an imprint of 300 Entertainment. YSL Records calls its roster of artists the "Slime Family." One of the YSL
+ Records founder\'s charges includes conspiracy to violate the Racketeer Influenced and Corrupt Organizations Act (RICO).', name='duckduckgo_search', id='afc
+cee2f-f36f-4d86-8abd-8d05d733649d', tool_call_id='6d073578-5bf1-449e-8d32-4f3b5f999a06'), HumanMessage(content='Retrieved document is not sufficient or relev
+ant to answer the query. Reformulate the query to search knowledge base again.', id='22af0af8-92d0-4876-ad4e-16457c37f1f3'), AIMessage(content='', additional
+_kwargs={'tool_calls': [ChatCompletionOutputToolCall(function=ChatCompletionOutputFunctionDefinition(arguments={'query': 'latest album by Young Thug'}, name=
+'duckduckgo_search', description=None), id='2315fb35-11ec-4a6b-be86-405a1efef411', type='function')]}, id='189ba925-9f82-4e9e-bdab-6dcfc49a196e', tool_calls=
+[{'name': 'duckduckgo_search', 'args': {'query': 'YSL Records founder name'}, 'id': 'cddeca43-88e2-4279-90aa-4c0b58b8b3c4', 'type': 'tool_call'}, {'name': 'd
+uckduckgo_search', 'args': {'query': 'latest album by Young Thug'}, 'id': '2315fb35-11ec-4a6b-be86-405a1efef411', 'type': 'tool_call'}]), ToolMessage(content
+='In 2016, Young Thug founded YSL Records, with its full name, "Young Stoner Life," perfectly representing the enigmatic persona of the label. YSL Records qu
+ickly became known for its ... Thug is also the founder of Young Stoner Life Records. However, the rapper was arrested in May 2022 and charged with conspirin
+g to violate the state\'s Racketeer Influenced and Corrupt Organizations (RICO) Act, according to. Prosecutors alleged that YSL was a street gang connected t
+o a sleuth of crimes. Prosecutors say YSL - the acronym for the artist\'s label, Young Stoner Life Records - also stands for Young Slime Life, an Atlanta-bas
+ed street gang affiliated with the national Bloods gang. Trontavious Stephens, a co-founder of YSL, took the witness stand for hours Wednesday and Thursday.
+Stephens testified that he had a criminal record, but he said he did not commit crimes with ... Young Thug\'s arrest sent shockwaves through the rap communit
+y. On May 9, 2022, the YSL rapper was apprehended along with 27 other alleged gang members as part of a sprawling 56-count indictment ...', name='duckduckgo_
+search', id='153464f0-aee1-4097-aae6-191ffa372aa6', tool_call_id='cddeca43-88e2-4279-90aa-4c0b58b8b3c4'), ToolMessage(content='The discography of American ra
+pper Young Thug consists of three studio albums, two compilation albums, twelve self-released mixtapes, seven commercial mixtapes, three extended plays, and
+sixty-nine singles (including 71 as a featured artist).. In 2015, Thug released his debut mixtape, Barter 6, which reached number 22 on the Billboard 200. Hi
+s 2016 mixtape, I\'m Up matched the same position. Atlanta singer (and Thugger\'s rumored girlfriend) also released her own "From a Woman" Friday. Even behin
+d bars, Young Thug remains prolific, with the rapper dropping his new single "From a Man ... Mariah The Scientist\'s new song "From A Woman" isn\'t musically
+ connected to "From A Man," though Mariah\'s cover art works as a flipside to Thug\'s. She sings about finding someone ... Future, Young Thug & More Join Mus
+tard On New Album \'Faith Of A Mustard Seed\': Stream. News | Jul 28, 2024, 11:00 AM PDT. HipHopDX brings you all the newest Young Thug albums, songs, and ..
+. Find top songs and albums by Young Thug including Lifestyle (feat. Young Thug & Rich Homie Quan), It\'s Up and more. ... JEFFREY to the country-ish sides o
+f Beautiful Thugger Girls—has cracked the mainstream, laying the groundwork for a new crop of fellow eccentrics like Lil Uzi Vert and Playboi Carti. In other 
+words, Thug hasn\'t adjusted ...', name='duckduckgo_search', id='79667433-d46e-4536-a04b-842223008bdc', tool_call_id='2315fb35-11ec-4a6b-be86-405a1efef411')
+, HumanMessage(content='Retrieved document is not sufficient or relevant to answer the query. Reformulate the query to search knowledge base again.', id='2f1
+b3773-8837-4f14-82b5-0a8bbccf3948'), HumanMessage(content='I don’t know.', id='a08edb17-a41a-458f-903e-0f7e15908e3f')]
diff --git a/comps/agent/langchain/requirements.txt b/comps/agent/langchain/requirements.txt
@@ -1,11 +1,11 @@
 # used by microservice
 docarray[full]
-
-#used by tools
-duckduckgo-search
 fastapi
 huggingface_hub
 langchain
+
+#used by tools
+langchain-google-community
 langchain-huggingface
 langchain-openai
 langchain_community