Merge Speculative Decoding fix for 2.12#118

Merged

Xaenalt merged 3 commits intoreleasefrom main

Aug 2, 2024

+201-85

Commits on Aug 1, 2024

[Frontend] New allowed_token_ids decoding request parameter (vllm-project#6753 )

njhill
authored and
Xaenalt
committed
[Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (vllm-project#6954 )

zifeitong
authored and
Xaenalt
committed

Commits on Aug 2, 2024

Merge pull request #117 from Xaenalt/bugfix/SamplingParams
Xaenalt
authored