youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,701 Commits 157 Branches 93 Tags
9cfa7697c1912fdb6e883eb5acea027e60fef733
Commit Graph

9701 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Cyrus Leung
9cfa7697c1 [V0 Deprecation] Enable the remaining multimodal tests in V1 (#25307) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
lirong
9fc86d2802 [Core] Enable sharded state loader for V1 engine and enhance test coverage (#25308) 
Signed-off-by: pengdrumli <pengdrumli@tencent.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Isotr0py
bc76128565 [Model] Cleanup InternViT's data parallel implementation (#25306) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Manoel Marques
af4dedf6d3 Generate _ModelInfo properties file when loading to improve loading speed (#23558) 
Signed-off-by: Manoel Marques <manoel.marques@ibm.com>
Signed-off-by: Manoel Marques <manoelmrqs@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Wenlong Wang
dad5f4d16d [Docs] Fix warnings in mkdocs build (continued) (#25042) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Michael Goin
c2fdc71c91 [CI Failure] Disable FlashInfer RoPE to unblock CI (#25299) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Cyrus Leung
e33af1e0c2 [V1] Support LLM.apply_model (#18465) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Roger Wang
0ac65d171b [Bugfix] Fix Qwen3-VL-MoE weight loading for EP (#25300) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Chen Zhang
267b4421b7 [Hybrid Allocator] Support full attention with different hidden size (#25101) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Cyrus Leung
8f3edbd93f [Optimization] Avoid repeated model architecture conversion for pooling models (#25261) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Chauncey
239aef5c9f [Bugfix] fix tool call arguments is empty (#25223) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: xin.li <xin.li@daocloud.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Chendi.Xue
9d70c103aa [BUG FIX][NON-CUDA]quick fix to avoid call cudagraph_unsafe in attention (#25298) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
d897924b45 [BugFix] Exclude self when checking for port collision (#25286) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
JartX
b7c986673d [BUGFIX] GPTQ quantization compatibility for Qwen3 Next MOE models (AutoGPTQ and AutoRound-GPTQ) (#25268) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Harry Mellor
14e1e9b09a Improve weight loading for encoder models in Transformers backend (#25289) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
ea01b17b6f [Misc] Support more collective_rpc return types (#25294) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
123e7ad492 [BugFix] Ensure appropriate guards in destructors (#25284) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Boyuan Feng
ce65ce2d61 [torch.compile] CUDAGraph Inductor partition integration (#24281) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Andrew Sansom
d4006bd84d [docs] Prompt Embedding feature support (#25288) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Andrew Sansom
7493472a9b test: Remove vestigial skip for prompt embeds tests after landing v1 Prompt Embeds support (#25291) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Maximilien de Bayser
937ab7e85e Don't skip special tokens with hermes-style tool calling (#25281) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Lucas Kabela
bc997c18ca [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
d55c6010ac [BugFix] Fix async scheduling CPU tensor race take 2 (#25279) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Lucia Fang
5051270200 allow disable flashinfer prefill (#25276) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Zhiyu
6e94161f94 Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Wentao Ye
e54a476058 [Compile] Fix Compile Warning for Ignoring MIN_BLOCK_PER_SM (#25193) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Alec S
8da7b98366 [Frontend] Responses API messages out, just harmony for now (#24985) 
Signed-off-by: Alec Solder <alecs@fb.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
David-Wen
9da51c77a9 Fix: Correct FusedMoE layer reference in auto_round quantization (#24818) 
Signed-off-by: David-Wen <18927700430@163.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
nvjullin
d0a1364188 [BugFix] Make FlashInferMetadataBuilder non-blocking (#25040) 
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Michael Goin
2c3ba7362f [Perf] Use FlashInfer RoPE for RotaryEmbedding.forward_cuda when available (#21126) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Harry Mellor
bfd32678e6 Specify platform in pip-compile pre-commit hook so it runs on MacOS (#25273) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
bnellnm
e29f599d30 [Bugfix] Fix chunked a2_scales in modular kernels (#25264) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Varun Sundar Rabindranath
b6724e95f8 [Bugfix] GPT OSS Attritbute error on H100 (#25228) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
qizixi
17b9f3a83d Optimize triton unified attention performance for sliding window attention (#24390) 
Signed-off-by: zixi-qi <qizixi@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Or Ozeri
378c68bead [KV offload][4/N] Offloading KV connector (#22595) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Lucia Fang
67f0418b1d [bugfix] fix structured outputs key missing issue from #24929 (#25195) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
samzong
779ed75310 [Docs] add __init__.py to vllm/model_executor/layers/quantization/compressed_tensors/transform (#24974) 
Signed-off-by: samzong <samzong.lu@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Wentao Ye
abb448b457 Update vllm/model_executor/layers/quantization/kernels/scaled_mm/cutlass.py 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
yewentao256
ae36150ec2 test 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:52 -07:00
Jialin Ouyang
2506ce5189 [Core][Prefix Hash] Fix prefix hash metrics sliding window maintainance (#24990) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-09-19 12:22:53 -06:00
Chauncey
47fd08aaf9 [CI/Build] fix test function_calling (#25072) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-19 12:16:32 -06:00
Harry Mellor
12aed7e453 Encoder model support for the Transformers backend (#25174) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 19:15:22 +01:00
LJH-LBJ
d90e212a3a Remove Redundant Assignment in Qwen3_VisionPatchMerger (#25224) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-19 12:15:13 -06:00
Jee Jee Li
2821986450 [Core] Modify the initialization parameters of the lora manager (#25249) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-19 18:01:28 +00:00
Cyrus Leung
6c117cff7d [Frontend] Pass API server count to each process (#23717) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 01:15:19 +08:00
Or Ozeri
7ac67ea525 [KV offload][3/N] Add worker-side CPU support (#21448) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-09-19 09:53:45 -07:00
samzong
ce75e15373 refactor(benchmarks): add type annotations to wait_for_endpoint parameters (#25218) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-19 16:36:52 +00:00
Harry Mellor
aed16879a9 Move ModelConfig from config/__init__.py to config/model.py (#25252) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 16:22:33 +00:00
Harry Mellor
cf278ff3b2 Update CODEOWNERS (#25269) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 09:12:55 -07:00
Icey
838d7116ba [Qwen] Remove cuda hard-code in qwen3 next (#25243) 
Signed-off-by: Icey <1790571317@qq.com>
 2025-09-19 12:25:12 +00:00