[AMD][Build] Porting dockerfiles from the ROCm/vllm fork #11777
Conversation
Signed-off-by: Gregory Shtrasberg <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
Signed-off-by: Gregory Shtrasberg <[email protected]>
mergify
bot
added
documentation
gshtras
force-pushed
the
dockerfile_unification
branch
from
8016502 to
a309129
Compare
Signed-off-by: Gregory Shtrasberg <[email protected]>
Dockerfile.rocm
Outdated
| # Make sure punica kernels are built (for LoRA) | ||
| ENV VLLM_INSTALL_PUNICA_KERNELS=1 |
There was a problem hiding this comment.
These kernels have been replaced by triton, so the env var isn't needed.
Dockerfile.rocm
Outdated
| FROM base AS fetch_vllm_0 | ||
| ONBUILD COPY ./ vllm/ | ||
| FROM base AS fetch_vllm_1 | ||
| ARG VLLM_REPO="https://github.com/ROCm/vllm.git" |
There was a problem hiding this comment.
I can understand wanting to get the build process to be similar between upstream and the rocm fork, however it seems counterintuitive to use the rocm fork by default for the dockerfile. Is it the case that this dockerfile will not work well for upstream vllm?
There was a problem hiding this comment.
Good catch, copy-paste error
Signed-off-by: Gregory Shtrasberg <[email protected]>
An attempt to unify the build process with how it is done in ROCm/vllm
Split the build process into:
In addition to not building the libraries each time, the new base image is now much smaller (7GB on docker hub vs 18GB previously), which allows to make the build process much faster (~10 minutes)