youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,695 Commits 157 Branches 93 Tags
e33af1e0c2ba9d8a5ddffa73db11e327cf691abe
Commit Graph

9695 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Cyrus Leung
e33af1e0c2 [V1] Support LLM.apply_model (#18465) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Roger Wang
0ac65d171b [Bugfix] Fix Qwen3-VL-MoE weight loading for EP (#25300) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Chen Zhang
267b4421b7 [Hybrid Allocator] Support full attention with different hidden size (#25101) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Cyrus Leung
8f3edbd93f [Optimization] Avoid repeated model architecture conversion for pooling models (#25261) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Chauncey
239aef5c9f [Bugfix] fix tool call arguments is empty (#25223) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: xin.li <xin.li@daocloud.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Chendi.Xue
9d70c103aa [BUG FIX][NON-CUDA]quick fix to avoid call cudagraph_unsafe in attention (#25298) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
d897924b45 [BugFix] Exclude self when checking for port collision (#25286) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
JartX
b7c986673d [BUGFIX] GPTQ quantization compatibility for Qwen3 Next MOE models (AutoGPTQ and AutoRound-GPTQ) (#25268) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Harry Mellor
14e1e9b09a Improve weight loading for encoder models in Transformers backend (#25289) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
ea01b17b6f [Misc] Support more collective_rpc return types (#25294) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
123e7ad492 [BugFix] Ensure appropriate guards in destructors (#25284) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Boyuan Feng
ce65ce2d61 [torch.compile] CUDAGraph Inductor partition integration (#24281) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Andrew Sansom
d4006bd84d [docs] Prompt Embedding feature support (#25288) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Andrew Sansom
7493472a9b test: Remove vestigial skip for prompt embeds tests after landing v1 Prompt Embeds support (#25291) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Maximilien de Bayser
937ab7e85e Don't skip special tokens with hermes-style tool calling (#25281) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Lucas Kabela
bc997c18ca [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
d55c6010ac [BugFix] Fix async scheduling CPU tensor race take 2 (#25279) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Lucia Fang
5051270200 allow disable flashinfer prefill (#25276) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Zhiyu
6e94161f94 Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Wentao Ye
e54a476058 [Compile] Fix Compile Warning for Ignoring MIN_BLOCK_PER_SM (#25193) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Alec S
8da7b98366 [Frontend] Responses API messages out, just harmony for now (#24985) 
Signed-off-by: Alec Solder <alecs@fb.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
David-Wen
9da51c77a9 Fix: Correct FusedMoE layer reference in auto_round quantization (#24818) 
Signed-off-by: David-Wen <18927700430@163.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
nvjullin
d0a1364188 [BugFix] Make FlashInferMetadataBuilder non-blocking (#25040) 
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Michael Goin
2c3ba7362f [Perf] Use FlashInfer RoPE for RotaryEmbedding.forward_cuda when available (#21126) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Harry Mellor
bfd32678e6 Specify platform in pip-compile pre-commit hook so it runs on MacOS (#25273) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
bnellnm
e29f599d30 [Bugfix] Fix chunked a2_scales in modular kernels (#25264) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Varun Sundar Rabindranath
b6724e95f8 [Bugfix] GPT OSS Attritbute error on H100 (#25228) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
qizixi
17b9f3a83d Optimize triton unified attention performance for sliding window attention (#24390) 
Signed-off-by: zixi-qi <qizixi@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Or Ozeri
378c68bead [KV offload][4/N] Offloading KV connector (#22595) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Lucia Fang
67f0418b1d [bugfix] fix structured outputs key missing issue from #24929 (#25195) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
samzong
779ed75310 [Docs] add __init__.py to vllm/model_executor/layers/quantization/compressed_tensors/transform (#24974) 
Signed-off-by: samzong <samzong.lu@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Wentao Ye
abb448b457 Update vllm/model_executor/layers/quantization/kernels/scaled_mm/cutlass.py 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
yewentao256
ae36150ec2 test 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Jialin Ouyang
2506ce5189 [Core][Prefix Hash] Fix prefix hash metrics sliding window maintainance (#24990) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-09-19 12:22:53 -06:00
Chauncey
47fd08aaf9 [CI/Build] fix test function_calling (#25072) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-19 12:16:32 -06:00
Harry Mellor
12aed7e453 Encoder model support for the Transformers backend (#25174) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 19:15:22 +01:00
LJH-LBJ
d90e212a3a Remove Redundant Assignment in Qwen3_VisionPatchMerger (#25224) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-19 12:15:13 -06:00
Jee Jee Li
2821986450 [Core] Modify the initialization parameters of the lora manager (#25249) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-19 18:01:28 +00:00
Cyrus Leung
6c117cff7d [Frontend] Pass API server count to each process (#23717) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 01:15:19 +08:00
Or Ozeri
7ac67ea525 [KV offload][3/N] Add worker-side CPU support (#21448) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-09-19 09:53:45 -07:00
samzong
ce75e15373 refactor(benchmarks): add type annotations to wait_for_endpoint parameters (#25218) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-19 16:36:52 +00:00
Harry Mellor
aed16879a9 Move ModelConfig from config/__init__.py to config/model.py (#25252) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 16:22:33 +00:00
Harry Mellor
cf278ff3b2 Update CODEOWNERS (#25269) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 09:12:55 -07:00
Icey
838d7116ba [Qwen] Remove cuda hard-code in qwen3 next (#25243) 
Signed-off-by: Icey <1790571317@qq.com>
 2025-09-19 12:25:12 +00:00
Cyrus Leung
5089fd749c [V0 Deprecation] Remove V0 logic from get_input_embeddings interface (#25242) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-19 11:10:52 +00:00
Nicolò Lucchesi
a3d087adec [P/D][Nixl] Introduce KVTransferMetrics and aggregation strategy (#22188) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-19 11:09:14 +00:00
Harry Mellor
058525b997 Move PoolerConfig from config/__init__.py to config/pooler.py (#25181) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 11:02:55 +00:00
Roger Wang
1dfea5f4a9 [Bugfix][Perf] Misc fixes for Qwen3 VL (#25238) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-19 10:46:16 +00:00
Isotr0py
cea91a32f2 [Kernel][Performance] Add Triton kernel for Qwen3-VL interleaved MRoPE (#25055) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-19 10:27:49 +00:00
Yan Ma
a684c0124c [bugfix] fix MHA for models like OpenGVLab/InternVL3_5-38B (#25146) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-19 08:45:06 +00:00