Add Workflow Executor Example (#892)

Signed-off-by: JoshuaL3000 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

WorkflowExecAgent/README.md

@@ -0,0 +1,131 @@
+# Workflow Executor Agent
+
+## Overview
+
+GenAI Workflow Executor Example showcases the capability to handle data/AI workflow operations via LangChain agents to execute custom-defined workflow-based tools. These workflow tools can be interfaced from any 3rd-party tools in the market (no-code/low-code/IDE) such as Alteryx, RapidMiner, Power BI, Intel Data Insight Automation which allows users to create complex data/AI workflow operations for different use-cases.
+
+### Workflow Executor
+
+This example demonstrates a single React-LangGraph with a `Workflow Executor` tool to ingest a user prompt to execute workflows and return an agent reasoning response based on the workflow output data.
+
+First the LLM extracts the relevant information from the user query based on the schema of the tool in `tools/tools.yaml`. Then the agent sends this `AgentState` to the `Workflow Executor` tool.
+
+`Workflow Executor` tool uses `EasyDataSDK` class as seen under `tools/sdk.py` to interface with several high-level API's. There are 3 steps to this tool implementation:
+
+1. Starts the workflow with workflow parameters and workflow id extracted from the user query.
+
+2. Periodically checks the workflow status for completion or failure. This may be through a database which stores the current status of the workflow
+
+3. Retrieves the output data from the workflow through a storage service.
+
+The `AgentState` is sent back to the LLM for reasoning. Based on the output data, the LLM generates a response to answer the user's input prompt.
+
+Below shows an illustration of this flow:
+
+![image](https://github.com/user-attachments/assets/cb135042-1505-4aef-8822-c78c2f72aa2a)
+
+### Workflow Serving for Agent
+
+As an example, here we have a Churn Prediction use-case workflow as the serving workflow for the agent execution. It is created through Intel Data Insight Automation platform. The image below shows a snapshot of the Churn Prediction workflow.
+
+![image](https://github.com/user-attachments/assets/c067f8b3-86cf-4abc-a8bd-51a98de8172d)
+
+The workflow contains 2 paths which can be seen in the workflow illustrated, the top path and bottom path.
+
+1. Top path - The training path which ends at the random forest classifier node is the training path. The data is cleaned through a series of nodes and used to train a random forest model for prediction.
+
+2. Bottom path - The inference path where trained random forest model is used for inferencing based on input parameter.
+
+For this agent workflow execution, the inferencing path is executed to yield the final output result of the `Model Predictor` node. The same output is returned to the `Workflow Executor` tool through the `Langchain API Serving` node.
+
+There are `Serving Parameters` in the workflow, which are the tool input variables used to start a workflow instance obtained from `params` the LLM extracts from the user query. Below shows the parameter configuration option for the Intel Data Insight Automation workflow UI.
+
+![image](https://github.com/user-attachments/assets/ce8ef01a-56ff-4278-b84d-b6e4592b28c6)
+
+Manually running the workflow yields the tabular data output as shown below:
+
+![image](https://github.com/user-attachments/assets/241c1aba-2a24-48da-8005-ec7bfe657179)
+
+In the workflow serving for agent, this output will be returned to the `Workflow Executor` tool. The LLM can then answer the user's original question based on this output.
+
+To start prompting the agent microservice, we will use the following command for this use case:
+
+```sh
+curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
+     "query": "I have a data with gender Female, tenure 55, MonthlyAvgCharges 103.7. Predict if this entry will churn. My workflow id is '${workflow_id}'."
+    }'
+```
+
+The user has to provide a `workflow_id` and workflow `params` in the query. `workflow_id` a unique id used for serving the workflow to the microservice. Notice that the `query` string includes all the workflow `params` which the user has defined in the workflow. The LLM will extract these parameters into a dictionary format for the workflow `Serving Parameters` as shown below:
+
+```python
+params = {"gender": "Female", "tenure": 55, "MonthlyAvgCharges": 103.7}
+```
+
+These parameters will be passed into the `Workflow Executor` tool to start the workflow execution of specified `workflow_id`. Thus, everything will be handled via the microservice.
+
+And finally here are the results from the microservice logs:
+
+![image](https://github.com/user-attachments/assets/969fefb7-543d-427f-a56c-dc70e474ae60)
+
+## Microservice Setup
+
+### Start Agent Microservice
+
+Workflow Executor will have a single docker image. First, build the agent docker image.
+
+```sh
+git clone https://github.com/opea-project/GenAIExamples.git
+cd GenAIExamples//WorkflowExecAgent/docker_image_build/
+docker compose -f build.yaml build --no-cache
+```
+
+Configure `GenAIExamples/WorkflowExecAgent/docker_compose/.env` file with the following. Replace the variables according to your usecase.
+
+```sh
+export SDK_BASE_URL=${SDK_BASE_URL}
+export SERVING_TOKEN=${SERVING_TOKEN}
+export HUGGINGFACEHUB_API_TOKEN=${HF_TOKEN}
+export llm_engine=${llm_engine}
+export llm_endpoint_url=${llm_endpoint_url}
+export ip_address=$(hostname -I | awk '{print $1}')
+export model="mistralai/Mistral-7B-Instruct-v0.3"
+export recursion_limit=${recursion_limit}
+export temperature=0
+export max_new_tokens=1000
+export WORKDIR=${WORKDIR}
+export TOOLSET_PATH=$WORKDIR/GenAIExamples/WorkflowExecAgent/tools/
+export http_proxy=${http_proxy}
+export https_proxy=${https_proxy}
+```
+
+Launch service by running the docker compose command.
+
+```sh
+cd $WORKDIR/GenAIExamples/WorkflowExecAgent/docker_compose
+docker compose -f compose.yaml up -d
+```
+
+### Validate service
+
+The microservice logs can be viewed using:
+
+```sh
+docker logs workflowexec-agent-endpoint
+```
+
+You should be able to see "HTTP server setup successful" upon successful startup.
+
+You can validate the service using the following command:
+
+```sh
+curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
+     "query": "I have a data with gender Female, tenure 55, MonthlyAvgCharges 103.7. Predict if this entry will churn. My workflow id is '${workflow_id}'."
+    }'
+```
+
+Update the `query` with the workflow parameters, workflow id, etc based on the workflow context.
+
+## Roadmap
+
+Phase II: Agent memory integration to enable capability to store tool intermediate results, such as workflow instance key.

WorkflowExecAgent/docker_compose/intel/cpu/xeon/compose_vllm.yaml

@@ -0,0 +1,31 @@
+# Copyright (C) 2024 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+services:
+  worflowexec-agent:
+    image: opea/agent-langchain:latest
+    container_name: workflowexec-agent-endpoint
+    volumes:
+      - ${WORKDIR}/GenAIComps/comps/agent/langchain/:/home/user/comps/agent/langchain/
+      - ${TOOLSET_PATH}:/home/user/tools/
+    ports:
+      - "9090:9090"
+    ipc: host
+    environment:
+      ip_address: ${ip_address}
+      strategy: react_langgraph
+      recursion_limit: ${recursion_limit}
+      llm_engine: ${llm_engine}
+      llm_endpoint_url: ${llm_endpoint_url}
+      model: ${model}
+      temperature: ${temperature}
+      max_new_tokens: ${max_new_tokens}
+      streaming: false
+      tools: /home/user/tools/tools.yaml
+      no_proxy: ${no_proxy}
+      http_proxy: ${http_proxy}
+      https_proxy: ${https_proxy}
+      port: 9090
+      SDK_BASE_URL: ${SDK_BASE_URL}
+      SERVING_TOKEN: ${SERVING_TOKEN}
+      custom_prompt: /home/user/tools/custom_prompt.py

WorkflowExecAgent/docker_image_build/build.yaml

@@ -0,0 +1,13 @@
+# Copyright (C) 2024 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+services:
+  agent-langchain:
+    build:
+      context: GenAIComps
+      dockerfile: comps/agent/langchain/Dockerfile
+      args:
+        http_proxy: ${http_proxy}
+        https_proxy: ${https_proxy}
+        no_proxy: ${no_proxy}
+    image: ${REGISTRY:-opea}/agent-langchain:${TAG:-latest}

WorkflowExecAgent/tests/1_build_images.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+# Copyright (C) 2024 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+set -e
+WORKPATH=$(dirname "$PWD")
+export WORKDIR=$WORKPATH/../../
+echo "WORKDIR=${WORKDIR}"
+
+function get_genai_comps() {
+    if [ ! -d "GenAIComps" ] ; then
+        git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout "${opea_branch:-"main"}" && cd ../
+    fi
+}
+
+function build_agent_docker_image() {
+    cd $WORKDIR/GenAIExamples/WorkflowExecAgent/docker_image_build/
+    get_genai_comps
+    echo "Build agent image with --no-cache..."
+    docker compose -f build.yaml build --no-cache
+}
+
+function main() {
+    echo "==================== Build agent docker image ===================="
+    build_agent_docker_image
+    echo "==================== Build agent docker image completed ===================="
+}
+
+main

WorkflowExecAgent/tests/2_start_vllm_service.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+# Copyright (C) 2024 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+set -e
+
+WORKPATH=$(dirname "$PWD")
+LOG_PATH="$WORKPATH/tests"
+vllm_port=${vllm_port}
+[[ -z "$vllm_port" ]] && vllm_port=8084
+model=mistralai/Mistral-7B-Instruct-v0.3
+export WORKDIR=$WORKPATH/../../
+export HF_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
+
+function build_vllm_docker_image() {
+    echo "Building the vllm docker images"
+    cd $WORKPATH
+    echo $WORKPATH
+    if [ ! -d "./vllm" ]; then
+        git clone https://github.com/vllm-project/vllm.git
+        cd ./vllm; git checkout tags/v0.6.0
+    else
+        cd ./vllm
+    fi
+    docker build -f Dockerfile.cpu -t vllm-cpu-env --shm-size=100g .
+    if [ $? -ne 0 ]; then
+        echo "opea/vllm:cpu failed"
+        exit 1
+    else
+        echo "opea/vllm:cpu successful"
+    fi
+}
+
+function start_vllm_service() {
+    echo "start vllm service"
+    docker run -d -p ${vllm_port}:${vllm_port} --rm --network=host --name test-comps-vllm-service -v ~/.cache/huggingface:/root/.cache/huggingface -v ${WORKPATH}/tests/tool_chat_template_mistral_custom.jinja:/root/tool_chat_template_mistral_custom.jinja -e HF_TOKEN=$HF_TOKEN -e http_proxy=$http_proxy -e https_proxy=$https_proxy -it vllm-cpu-env --model ${model} --port ${vllm_port} --chat-template /root/tool_chat_template_mistral_custom.jinja --enable-auto-tool-choice --tool-call-parser mistral
+    echo ${LOG_PATH}/vllm-service.log
+    sleep 5s
+    echo "Waiting vllm ready"
+    n=0
+    until [[ "$n" -ge 100 ]] || [[ $ready == true ]]; do
+        docker logs test-comps-vllm-service &> ${LOG_PATH}/vllm-service.log
+        n=$((n+1))
+        if grep -q "Uvicorn running on" ${LOG_PATH}/vllm-service.log; then
+            break
+        fi
+        if grep -q "No such container" ${LOG_PATH}/vllm-service.log; then
+            echo "container test-comps-vllm-service not found"
+            exit 1
+        fi
+        sleep 5s
+    done
+    sleep 5s
+    echo "Service started successfully"
+}
+
+function main() {
+    echo "==================== Build vllm docker image ===================="
+    build_vllm_docker_image
+    echo "==================== Build vllm docker image completed ===================="
+
+    echo "==================== Start vllm docker service ===================="
+    start_vllm_service
+    echo "==================== Start vllm docker service completed ===================="
+}
+
+main

WorkflowExecAgent/tests/3_launch_and_validate_agent.sh

@@ -0,0 +1,66 @@
+#!/bin/bash
+# Copyright (C) 2024 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+set -e
+
+WORKPATH=$(dirname "$PWD")
+workflow_id=9809
+vllm_port=${vllm_port}
+[[ -z "$vllm_port" ]] && vllm_port=8084
+export WORKDIR=$WORKPATH/../../
+echo "WORKDIR=${WORKDIR}"
+export SDK_BASE_URL=${SDK_BASE_URL}
+export SERVING_TOKEN=${SERVING_TOKEN}
+export HF_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
+export llm_engine=vllm
+export ip_address=$(hostname -I | awk '{print $1}')
+export llm_endpoint_url=http://${ip_address}:${vllm_port}
+export model=mistralai/Mistral-7B-Instruct-v0.3
+export recursion_limit=25
+export temperature=0
+export max_new_tokens=1000
+export TOOLSET_PATH=$WORKDIR/GenAIExamples/WorkflowExecAgent/tools/
+
+function start_agent_and_api_server() {
+    echo "Starting Agent services"
+    cd $WORKDIR/GenAIExamples/WorkflowExecAgent/docker_compose/intel/cpu/xeon
+    WORKDIR=$WORKPATH/docker_image_build/ docker compose -f compose_vllm.yaml up -d
+    echo "Waiting agent service ready"
+    sleep 5s
+}
+
+function validate() {
+    local CONTENT="$1"
+    local EXPECTED_RESULT="$2"
+    local SERVICE_NAME="$3"
+
+    if echo "$CONTENT" | grep -q "$EXPECTED_RESULT"; then
+        echo "[ $SERVICE_NAME ] Content is as expected: $CONTENT"
+        echo "[TEST INFO]: Workflow Executor agent service PASSED"
+    else
+        echo "[ $SERVICE_NAME ] Content does not match the expected result: $CONTENT"
+        echo "[TEST INFO]: Workflow Executor agent service FAILED"
+    fi
+}
+
+function validate_agent_service() {
+    echo "----------------Test agent ----------------"
+    local CONTENT=$(curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
+     "query": "I have a data with gender Female, tenure 55, MonthlyAvgCharges 103.7. Predict if this entry will churn. My workflow id is '${workflow_id}'."
+    }')
+    validate "$CONTENT" "The entry is not likely to churn" "workflowexec-agent-endpoint"
+    docker logs workflowexec-agent-endpoint
+}
+
+function main() {
+    echo "==================== Start agent ===================="
+    start_agent_and_api_server
+    echo "==================== Agent started ===================="
+
+    echo "==================== Validate agent service ===================="
+    validate_agent_service
+    echo "==================== Agent service validated ===================="
+}
+
+main

WorkflowExecAgent/tests/README.md

@@ -0,0 +1,38 @@
+# Validate Workflow Agent Microservice
+
+Microservice validation for Intel Data Insight Automation platform workflow serving.
+
+## Usage
+
+Configure necessary variables as listed below. Replace the variables according to your usecase.
+
+```sh
+export SDK_BASE_URL=${SDK_BASE_URL}
+export SERVING_TOKEN=${SERVING_TOKEN}
+export HUGGINGFACEHUB_API_TOKEN=${HF_TOKEN}
+export workflow_id=${workflow_id}       # workflow_id of the serving workflow
+export vllm_port=${vllm_port}           # vllm serving port
+export ip_address=$(hostname -I | awk '{print $1}')
+export VLLM_CPU_OMP_THREADS_BIND=${VLLM_CPU_OMP_THREADS_BIND}
+export http_proxy=${http_proxy}
+export https_proxy=${https_proxy}
+```
+
+Note: `SDK_BASE_URL` and `SERVING_TOKEN` can be obtained from Intel Data Insight Automation platform.
+
+Launch validation by running the following command.
+
+```sh
+cd GenAIExamples/WorkflowExecAgent/tests
+. /test_compose_on_xeon.sh
+```
+
+`test_compose_on_xeon.sh` will run the other `.sh` files under `tests/`. The validation script launches 1 docker container for the agent microservice, and another for the vllm model serving on CPU. When validation is completed, all containers will be stopped.
+
+The validation is tested by checking if the model reasoning output response matches a partial substring. The expected output is shown below:
+
+![image](https://github.com/user-attachments/assets/88081bc8-7b73-470d-970e-92e0fe5f96ec)
+
+## Note
+
+- Currently the validation test is only designed with vllm model serving (CPU only).