Sync with [email protected] #103

dtrifiro · 2024-07-23T08:44:56Z

Merge vllm-project/vllm:main@v0.5.3 into main

…ect#6289)

Co-authored-by: kevin <[email protected]>

Signed-off-by: kevin <[email protected]>

…eanup bug (vllm-project#6425) Co-authored-by: Mor Zusman <[email protected]>

…llm-project#6419)

vllm-project#5923)

Signed-off-by: Thomas Parnell <[email protected]>

…oject#6481) [doc][misc] remind users to cancel debugging environment variables after debugging (vllm-project#6481)

…t#6230)

…`v_scale` (vllm-project#6081)

…t#6410)

…m-project#6455) original title: [Distributed][Model] Rank-based Component Creation for Pipeline Parallelism Memory Optimization

Co-authored-by: caishangming.csm <[email protected]>

…ct#6431)

Signed-off-by: Muralidhar Andoorveedu <[email protected]>

)

…ct#6624)

Co-authored-by: Roger Wang <[email protected]>

…roject#6649)

…ject#6656) Signed-off-by: kevin <[email protected]>

…m-project#6234) Co-authored-by: Antoni Baum <[email protected]>

…oject#6596)

…vllm-project#6665)

…m-project#6666)

)

Co-authored-by: tianyi.zhao <[email protected]> Co-authored-by: youkaichao <[email protected]>

…ect#6528)

…oject#6529) [doc][distributed] add more doc for setting up multi-node environment (vllm-project#6529)

Merge [vllm-project/vllm](https://github.com/vllm-project/vllm):[main](https://github.com/vllm-project/vllm/tree/main)@[v0.5.3](https://github.com/vllm-project/vllm/commit/bb2fc08072db2d96e547407b4301fb6ba141d9d6) into main

openshift-ci · 2024-07-23T08:45:03Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dtrifiro

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dtrifiro]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

WoosukKwon and others added 30 commits July 15, 2024 11:58

[Docs] Add Google Cloud to sponsor list (vllm-project#6450)

3dee97b

[Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (vllm-proj…

ec9933f

…ect#6289)

[CI/Build][TPU] Add TPU CI test (vllm-project#6277)

4552e37

Co-authored-by: kevin <[email protected]>

Pin sphinx-argparse version (vllm-project#6453)

d6f3b3d

Signed-off-by: kevin <[email protected]>

[BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cl…

9ad32da

…eanup bug (vllm-project#6425) Co-authored-by: Mor Zusman <[email protected]>

[Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (v…

d92b3c5

…llm-project#6419)

[Docs] Announce 5th meetup (vllm-project#6458)

37d7766

[CI/Build] vLLM cache directory for images (vllm-project#6444)

d970115

[Frontend] Support for chat completions input in the tokenize endpoint (

7a3d2a5

vllm-project#5923)

[Misc] Fix typos in spec. decode metrics logging. (vllm-project#6470)

7508a3d

Signed-off-by: Thomas Parnell <[email protected]>

[Core] Use numpy to speed up padded token processing (vllm-project#6442)

2bb0489

[CI/Build] Remove "boardwalk" image asset (vllm-project#6460)

38ef948

[doc][misc] remind to cancel debugging environment variables (vllm-pr…

9f4ccec

…oject#6481) [doc][misc] remind users to cancel debugging environment variables after debugging (vllm-project#6481)

[Hardware][TPU] Support MoE with Pallas GMM kernel (vllm-project#6457)

c467dff

[Doc] Fix the lora adapter path in server startup script (vllm-projec…

94162be

…t#6230)

[Misc] Log spec decode metrics (vllm-project#6454)

160e1d8

[Kernel][Attention] Separate Attention.kv_scale into k_scale and …

978aed5

…`v_scale` (vllm-project#6081)

[ci][distributed] add pipeline parallel correctness test (vllm-projec…

09c2eb8

…t#6410)

[misc][distributed] improve tests (vllm-project#6488)

7f62077

[misc][distributed] add seed to dummy weights (vllm-project#6491)

ce37be7

[Distributed][PP] only create embedding & lm head when necessary (vll…

1d094fd

…m-project#6455) original title: [Distributed][Model] Rank-based Component Creation for Pipeline Parallelism Memory Optimization

[ROCm] Cleanup Dockerfile and remove outdated patch (vllm-project#6482)

1038388

[Misc][Speculative decoding] Typos and typing fixes (vllm-project#6467)

a19e8d3

Co-authored-by: caishangming.csm <[email protected]>

[Doc][CI/Build] Update docs and tests to use vllm serve (vllm-proje…

5bf35a9

…ct#6431)

[Bugfix] Fix for multinode crash on 4 PP (vllm-project#6495)

5fa6e98

Signed-off-by: Muralidhar Andoorveedu <[email protected]>

[TPU] Remove multi-modal args in TPU backend (vllm-project#6504)

e09ce75

[Misc] Use torch.Tensor for type annotation (vllm-project#6505)

a9a2e74

[Core] Refactor _prepare_model_input_tensors - take 2 (vllm-project#6164

2fa4623

)

[DOC] - Add docker image to Cerebrium Integration (vllm-project#6510)

a38524f

[Bugfix] Fix Ray Metrics API usage (vllm-project#6354)

5f0b993

ywang96 and others added 22 commits July 21, 2024 17:37

[Model] Initial Support for Chameleon (vllm-project#5770)

c9eef37

[Misc] Add a wrapper for torch.inference_mode (vllm-project#6618)

42de2ce

[Bugfix] Fix vocab_size field access in llava_next.py (vllm-proje…

89c1c6a

…ct#6624)

[Frontend] Refactor prompt processing (vllm-project#4028)

739b61a

Co-authored-by: Roger Wang <[email protected]>

[Bugfix][Kernel] Use int64_t for indices in fp8 quant kernels (vllm-p…

fea59c7

…roject#6649)

[ci] Use different sccache bucket for CUDA 11.8 wheel build (vllm-pro…

69d5ae3

…ject#6656) Signed-off-by: kevin <[email protected]>

[Core] Support dynamically loading Lora adapter from HuggingFace (vll…

42c7f66

…m-project#6234) Co-authored-by: Antoni Baum <[email protected]>

[ci][build] add back vim in docker (vllm-project#6661)

5a96ee5

[Misc] Remove deprecation warning for beam search (vllm-project#6659)

bdf5fd1

[Core] Modulize prepare input and attention metadata builder (vllm-pr…

e0c1575

…oject#6596)

[Bugfix] Fix null modules_to_not_convert in FBGEMM Fp8 quantization (…

c5e8330

…vllm-project#6665)

[Misc] Enable chunked prefill by default for long context models (vll…

729171a

…m-project#6666)

[misc] add start loading models for users information (vllm-project#6670

7c2749a

)

add tqdm when loading checkpoint shards (vllm-project#6569)

e519ae0

Co-authored-by: tianyi.zhao <[email protected]> Co-authored-by: youkaichao <[email protected]>

[Misc] Support FP8 kv cache scales from compressed-tensors (vllm-proj…

9e0b558

…ect#6528)

[doc][distributed] doc for setting up multi-node environment (vllm-pr…

c051bfe

…oject#6529) [doc][distributed] add more doc for setting up multi-node environment (vllm-project#6529)

[Misc] Manage HTTP connections in one place (vllm-project#6600)

97234be

[misc] only tqdm for first rank (vllm-project#6672)

c520124

[VLM][Model] Support image input for Chameleon (vllm-project#6633)

22fa2e3

support ignore patterns in model loader (vllm-project#6673)

3eda4ec

Bump version to v0.5.3 (vllm-project#6674)

bb2fc08

Sync with [email protected]

c2ef5a8

Merge [vllm-project/vllm](https://github.com/vllm-project/vllm):[main](https://github.com/vllm-project/vllm/tree/main)@[v0.5.3](https://github.com/vllm-project/vllm/commit/bb2fc08072db2d96e547407b4301fb6ba141d9d6) into main

dtrifiro added the code-sync Sync with upstream label Jul 23, 2024

openshift-ci bot requested review from joerunde and Xaenalt July 23, 2024 08:45

openshift-ci bot added the approved label Jul 23, 2024

dtrifiro mentioned this pull request Jul 23, 2024

Sync with [email protected] #102

Closed

dtrifiro merged commit b2dc37f into opendatahub-io:main Jul 23, 2024
13 of 17 checks passed

dtrifiro deleted the sync-with-upstream branch July 23, 2024 08:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sync with [email protected] #103

Sync with [email protected] #103

dtrifiro commented Jul 23, 2024

openshift-ci bot commented Jul 23, 2024

Sync with [email protected] #103

Sync with [email protected] #103

Conversation

dtrifiro commented Jul 23, 2024

openshift-ci bot commented Jul 23, 2024