Adapt to latest vllm changes #632
Conversation
lianhao
force-pushed
the
bug631
branch
2 times, most recently
from
4aad447 to
6bfddb6
Compare
There was a problem hiding this comment.
Approved, matches changes for "GenAIExamples" in opea-project/GenAIExamples#1210 (and corresponding PR for "GenAIComps" repo).
@poussa ?
- Remove --eager-enforce on hpu to improve performance - Refactor to the upstream docker entrypoint changes Fixes issue opea-project#631. Signed-off-by: Lianhao Lu <[email protected]>
|
Investigating the CI failure for "agent, gaudi, ci-gaudi-values, common" test, I see 2 bugs:
(Besides the size, I think another model would be nicer as default due to license used on Meta's models.) |
|
My vLLM PR includes same agent and (relevant) vLLM component changes as yours, but strangely that same CI agent test succeeded for it: https://github.com/opea-project/GenAIInfra/actions/runs/12262626198/job/34212355870?pr=610 ? EDIT: today's push on my PR got the same issue. |
There was a problem hiding this comment.
--tensor-parallel-size option can be dropped, as 1 value is the default:
https://docs.vllm.ai/en/latest/usage/engine_args.html
|
we need to wait for PR #642 to land-in first |
Description
Issues
Fixes #631.
Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
List the newly introduced 3rd party dependency if exists.
Tests
Describe the tests that you ran to verify your changes.