ChatQnA - add files for deploy ChatQnA application on AMD ROCm with vLLM service #1181
Conversation
…LLM service: 1. ChatQnA/docker_compose/amd/gpu/rocm-vllm/compose_vllm.yaml 2. ChatQnA/docker_compose/amd/gpu/rocm-vllm/set_env_vllm.sh 3. ChatQnA/docker_compose/amd/gpu/rocm-vllm/Dockerfile-vllm 4. ChatQnA/docker_compose/amd/gpu/rocm-vllm/REAMDE.md. 5. ChatQnA/tests/test_compose_on_rocm_vllm.sh Fix build.yaml and playwright.config.ts Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
chyundunovDatamonsters
requested a review
from lvliang-intel
as a code owner
| @@ -0,0 +1,18 @@ | |||
| FROM rocm/vllm:rocm6.2_mi300_ubuntu20.04_py3.9_vllm_0.6.4 | |||
There was a problem hiding this comment.
let's put the dockerfile under GenAIExamples/ChatQnA/, it can be named as Dockerfile.vllm_rocm.
There was a problem hiding this comment.
Good. I will place the file according to the proposed path and adapt the scripts
|
Let use one folder |
| @@ -0,0 +1,265 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
ChatQnA/tests/test_compose_on_rocm_vllm.sh --> ChatQnA/tests/test_compose_vllm_on_rocm.sh
The naming format will impact CI trigger.
There was a problem hiding this comment.
For this file name, rocm_vllm will be recognized as a hardware label, but we just need rocm. So it need to be update like what I commented.
There was a problem hiding this comment.
Accepted. Let's put everything in one place for the Arm architecture.
…oy app on AMD GPU Signed-off-by: Chingis Yundunov <[email protected]>
…oy app on AMD GPU Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
Description
Add files for deploy ChatQnA application on AMD ROCm with vLLM service:
ChatQnA/docker_compose/amd/gpu/rocm-vllm/compose_vllm.yaml - Docker Compose file
ChatQnA/docker_compose/amd/gpu/rocm-vllm/set_env_vllm.sh - set envs script
ChatQnA/docker_compose/amd/gpu/rocm-vllm/Dockerfile-vllm - Dockerfile for build vLLM Docker image for service on ROCm
ChatQnA/docker_compose/amd/gpu/rocm-vllm/README.md - README file
Issues
It was required to be able to deploy the ChatQnA application on AMD hardware using the vLLM Service
Type of change
Dependencies
Tests
Testing is performed by a script GenAIExamples/ChatQnA/tests/test_compose_on_rocm_vllm.sh