youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,748 Commits 157 Branches 93 Tags
6dbbecd5b2d90341db9c4d63a67d87c515efb0b8
Commit Graph

9748 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Luka Govedič
6dbbecd5b2 [torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Simon Mo
6850bfe15c [misc] Remove RFC review hours reference (#25416) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Lucia Fang
d988b84e8e [DP] support torchrun external launcher with Data Parallelism (#24899) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Michael Goin
7337ec6c9f [CI Failure] Fix fp8 kv cache on <SM90 (#25396) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
ElizaWszola
90ba32a0bf [Compiler] Disable Inductor standalone compile by default (#25391) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Daisy-Ma-coder
2a8bd2b93b [CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) 
Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Bowen Wang
3968ae72ed [EPLB] Reduce EPLB Inference Overhead (#24573) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Burkhard Ringlein
e55ffe3595 [V1][Attention] Split triton_attn in triton-only and rocm specific backends (#24648) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Csrayz
4057e2b162 [Bugfix] Fix several issues with p2p xPyD in GET type (#23993) 
Signed-off-by: Csrayz <jover@cmbchina.com>
Signed-off-by: ivyilike <pww123@cmbchina.com>
Co-authored-by: ivyilike <pww123@cmbchina.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Sara-KS
cc494282a9 [Kernel] MI-300X triton moe configs (#23445) 
Signed-off-by: Sara Kokkila Schumacher <saraks@ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Harry Mellor
44be2b7349 Make mypy behave like a proper pre-commit hook (#25313) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Harry Mellor
104e62fbc8 Make pickle import check fast (#25379) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Cyrus Leung
ddf4e1f56f [Misc] Remove unused encoder-decoder error strings (#25374) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Yizhou
cbba9bd0b0 refactor: abstract graph mode support into platform interface (#25161) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nicolò Lucchesi
4bc6b5d2c3 [TPU] Deprecate xm.mark_step in favor of `torch_xla.sync (#25254) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nicolò Lucchesi
8d8de42790 [TPU][Bugfix][CI] Fix broken tests/build dependency (#25255) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Eldar Kurtić
ef85a438da Enable Eagle3 speculative decoding for GPT-OSS model (#25246) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Cyrus Leung
2f237d3df4 [V0 Deprecation] Remove MultiModalPlaceholderMap (#25366) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Cyrus Leung
243c358fa8 [V0 Deprecation] Remove V0-only methods in multi-modal registry (#25362) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
WeiQing Chen
1b3aa0f297 [Bugfix] Fix hermes tool parser handling of non-string argument types (#22002) 
Signed-off-by: wangzi <3220100013@zju.edu.cn>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: wangzi <3220100013@zju.edu.cn>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
WeiQing Chen
dba6db9937 [Docs] GSM8K Accuracy Evaluation doc update (#25360) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Roger Wang
5322390f1d [Model] Support Dots OCR (#24645) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: yinz-aizip <yinz@aizip.ai>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Deboleina
5f6a36054a Multimodal - audio tests (#25285) 
Signed-off-by: Debolina Roy <debroy@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Yang Liu
e348e1027c [Bugfix][V0 Deprecation][CI] use async mock and await for async method (#25325) 
Signed-off-by: Yang <lymailforjob@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
a815d820ee Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Isotr0py
319966a678 [Perf] Further optimization for Qwen3-VL fast_pos_embed_interpolate (#25347) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
b81364a7cd [V0 Deprecation] Remove V0 sampling metadata (#25345) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Rahul Tuli
791089df20 feat: Enable engine-level arguments with speculators models (#25250) 
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
71f2b5ddea [V0 Deprecation] Remove async_output_proc, preemption mode, delay factor (#25334) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
81e17a1e26 [V0 Deprecation] Remove V0 Sequence class & Sampler (#25332) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
yewentao256
ed84bda7a5 fix cub helpers 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
yewentao256
c7b1c0cf8b fix cub_helpers 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Cyrus Leung
a31d353b71 [Optimization] Cache chat template result when processor fails to be loaded (#25341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Simon Danielsson
80cad257da [Bugfix] Typos in error message for missing model config file (#25339) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Roger Wang
5fd95c77af [MM][Perf] Minor Optimization on Qwen3-VL fast_pos_embed_interpolate (#25337) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Isotr0py
f6278e3065 [V1] Add sliding window support to Flex Attention backend (#24089) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
9e9b3b4ff9 [V0 Deprecation] Remove V0 MP executor (#25329) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
20235c1822 [V0 Deprecation] Remove from_seq_group methods (#25330) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Wenlong Wang
059a13a3bc [Multi Modal][Performance] Fused Q,K's apply_rope in more models (#25005) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
a6cf307fa8 [V0 Deprecation] Remove V0 model runner base & simplify worker base (#25328) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Huamin Li
b18dde7478 [Doc] improve test-pipeline.yaml documentation (#25305) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
7cdd90211b [V0 Deprecation] Remove V0 core (#25321) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
86fdd686be [CI] Skip tests failing on main (#25326) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
171592330b [Chore] Remove unused sampler in models (#25324) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
4bb2eb42d4 [V0 Deprecation] Remove V0 Output Processor (#25320) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Woosuk Kwon
32d43a5a9e [V0 Deprecation] Remove LLMEngine (#25033) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Michael Yao
d9ba479eee [Docs] Fix warnings in vllm/profiler and vllm/transformers_utils (#25220) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Cyrus Leung
9cfa7697c1 [V0 Deprecation] Enable the remaining multimodal tests in V1 (#25307) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
lirong
9fc86d2802 [Core] Enable sharded state loader for V1 engine and enhance test coverage (#25308) 
Signed-off-by: pengdrumli <pengdrumli@tencent.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Isotr0py
bc76128565 [Model] Cleanup InternViT's data parallel implementation (#25306) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00