Code

(DO NOT MERGE) 6587 fix#101

Code

Closed

prashantgupta24 wants to merge 50 commits intomainopendatahub-io/vllm:mainfrom 6587-fixopendatahub-io/vllm:6587-fixCopy head branch name to clipboard

+5,323-2,393

Commits

Commits on Jul 19, 2024

• [Docs] Add Google Cloud to sponsor list (vllm-project#6450)

  

  WoosukKwon

  authored and

  fialhocoelho

  committed
  3858045

• [Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (vllm-project#6289)

  

  WoosukKwon

  authored and

  fialhocoelho

  committed
  92ae626

• [CI/Build][TPU] Add TPU CI test (vllm-project#6277)

  Show description for f8e8321
  

  2 peopleauthored and

  fialhocoelho

  committed
  f8e8321

• Pin sphinx-argparse version (vllm-project#6453)

  Show description for 21181b6
  

  khluu

  authored and

  fialhocoelho

  committed
  21181b6

• [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (vllm-project#6425)

  Show description for 9d2dde3
  

  2 peopleauthored and

  fialhocoelho

  committed
  9d2dde3

• [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (vllm-project#6419)

  

  g-eoj

  authored and

  fialhocoelho

  committed
  a611890

• [Docs] Announce 5th meetup (vllm-project#6458)

  

  WoosukKwon

  authored and

  fialhocoelho

  committed
  6b0c7f8

• [CI/Build] vLLM cache directory for images (vllm-project#6444)

  

  DarkLight1337

  authored and

  fialhocoelho

  committed
  e9e312b

• [Frontend] Support for chat completions input in the tokenize endpoint (vllm-project#5923)

  

  sasha0552

  authored and

  fialhocoelho

  committed
  f55cf50

• [Misc] Fix typos in spec. decode metrics logging. (vllm-project#6470)

  Show description for 93db3e5
  

  tdoublep

  authored and

  fialhocoelho

  committed
  93db3e5

• [Core] Use numpy to speed up padded token processing (vllm-project#6442)

  

  peng1999

  authored and

  fialhocoelho

  committed
  d49020b

• [CI/Build] Remove "boardwalk" image asset (vllm-project#6460)

  

  DarkLight1337

  authored and

  fialhocoelho

  committed
  7ce90fd

• [doc][misc] remind to cancel debugging environment variables (vllm-project#6481)

  Show description for 75275b3
  

  youkaichao

  authored and

  fialhocoelho

  committed
  75275b3

• [Hardware][TPU] Support MoE with Pallas GMM kernel (vllm-project#6457)

  

  WoosukKwon

  authored and

  fialhocoelho

  committed
  c2d4696

• [Doc] Fix the lora adapter path in server startup script (vllm-project#6230)

  

  Jeffwan

  authored and

  fialhocoelho

  committed
  e6b88ce

• [Misc] Log spec decode metrics (vllm-project#6454)

  

  comaniac

  authored and

  fialhocoelho

  committed
  3d4d6d9

• [Kernel][Attention] Separate Attention.kv_scale into k_scale and v_scale (vllm-project#6081)

  

  mgoin

  authored and

  fialhocoelho

  committed
  fdd5404

• [ci][distributed] add pipeline parallel correctness test (vllm-project#6410)

  

  youkaichao

  authored and

  fialhocoelho

  committed
  2219bb5

• [misc][distributed] improve tests (vllm-project#6488)

  

  youkaichao

  authored and

  fialhocoelho

  committed
  572aff2

• [misc][distributed] add seed to dummy weights (vllm-project#6491)

  

  youkaichao

  authored and

  fialhocoelho

  committed
  7668b56

• [Distributed][PP] only create embedding & lm head when necessary (vllm-project#6455)

  Show description for b63d466
  

  wushidonguc

  authored and

  fialhocoelho

  committed
  b63d466

• [ROCm] Cleanup Dockerfile and remove outdated patch (vllm-project#6482)

  

  hongxiayang

  authored and

  fialhocoelho

  committed
  1296b98

• [Misc][Speculative decoding] Typos and typing fixes (vllm-project#6467)

  Show description for 5e979fb
  

  ShangmingCai

  authored and

  fialhocoelho

  committed
  5e979fb

• [Doc][CI/Build] Update docs and tests to use vllm serve (vllm-project#6431)

  

  DarkLight1337

  authored and

  fialhocoelho

  committed
  34ac3ed

• [Bugfix] Fix for multinode crash on 4 PP (vllm-project#6495)

  Show description for 30087e9
  

  andoorve

  authored and

  fialhocoelho

  committed
  30087e9

• [TPU] Remove multi-modal args in TPU backend (vllm-project#6504)

  

  WoosukKwon

  authored and

  fialhocoelho

  committed
  993cc28

• [Misc] Use torch.Tensor for type annotation (vllm-project#6505)

  

  WoosukKwon

  authored and

  fialhocoelho

  committed
  4abbb3c

• [Core] Refactor _prepare_model_input_tensors - take 2 (vllm-project#6164)

  

  comaniac

  authored and

  fialhocoelho

  committed
  d380e55

• [DOC] - Add docker image to Cerebrium Integration (vllm-project#6510)

  

  milo157

  authored and

  fialhocoelho

  committed
  b97793d

• [Bugfix] Fix Ray Metrics API usage (vllm-project#6354)

  

  Yard1

  authored and

  fialhocoelho

  committed
  3c4a725

• [Core] draft_model_runner: Implement prepare_inputs on GPU for advance_step (vllm-project#6338)

  

  alexm-neuralmagic

  authored and

  fialhocoelho

  committed
  d9124a4

• [ Kernel ] FP8 Dynamic-Per-Token Quant Kernel (vllm-project#6511)

  Show description for 6834854
  

  2 peopleauthored and

  fialhocoelho

  committed
  6834854

• [Model] Pipeline parallel support for Mixtral (vllm-project#6516)

  

  comaniac

  authored and

  fialhocoelho

  committed
  e57a59c

• [ Kernel ] Fp8 Channelwise Weight Support (vllm-project#6487)

  

  robertgshaw2-neuralmagic

  authored and

  fialhocoelho

  committed
  e7aecdc

• [core][model] yet another cpu offload implementation (vllm-project#6496)

  Show description for e5ce3af
  

  2 peopleauthored and

  fialhocoelho

  committed
  e5ce3af

• [BugFix] Avoid secondary error in ShmRingBuffer destructor (vllm-project#6530)

  

  njhill

  authored and

  fialhocoelho

  committed
  0945568

• [Core] Introduce SPMD worker execution using Ray accelerated DAG (vllm-project#6032)

  Show description for 91cec1b
  

  2 peopleauthored and

  fialhocoelho

  committed
  91cec1b

• [Misc] Minor patch for draft model runner (vllm-project#6523)

  

  comaniac

  authored and

  fialhocoelho

  committed
  bd40d82

• [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (vllm-project#6227)

  Show description for 1a1ffe8
  

  2 peopleauthored and

  fialhocoelho

  committed
  1a1ffe8

• [Bugfix] Update flashinfer.py with PagedAttention forwards - Fixes Gemma2 OpenAI Server Crash (vllm-project#6501)

  

  noamgat

  authored and

  fialhocoelho

  committed
  e9b1df5

• [TPU] Refactor TPU worker & model runner (vllm-project#6506)

  

  WoosukKwon

  authored and

  fialhocoelho

  committed
  d49a5e3

• [ Misc ] Improve Min Capability Checking in compressed-tensors (vllm-project#6522)

  

  robertgshaw2-neuralmagic

  authored and

  fialhocoelho

  committed
  d49092f

• [ci] Reword Github bot comment (vllm-project#6534)

  

  khluu

  authored and

  fialhocoelho

  committed
  70f7a37

• Squash 6034

  Show description for 2718aea
  fialhocoelho

  committed
  2718aea

• Squash 6357

  Show description for 07eba1d
  fialhocoelho

  committed
  07eba1d

• Merge squash from allowed_token_ids branch

  Show description for 9ddd0b7
  fialhocoelho

  committed
  9ddd0b7

• Squash 6587

  prashantgupta24

  committed
  817974a

Commits on Jul 20, 2024

• Update Dockerfile.ubi to install vllm-tgis-adapter from fix branch

  njhill

  committed
  9b0ca0d

Commits on Jul 22, 2024

• Update Dockerfile.ubu to install vllm-tgis-adapter from main branch

  njhill

  committed
  9ab1931

Commits on Jul 23, 2024

• Add --system-site-packages to virtual env in image

  njhill

  committed
  6f83065