|
|
f68f7ee030
|
Revert "[nit]: Fix import for the lmcache integration (#27600)"
This reverts commit 05e034f085.
|
2025-10-28 13:46:05 -07:00 |
|
|
|
141e6a0505
|
[Misc] Make reorder batch also separate extends (#27367)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-10-28 10:55:10 -07:00 |
|
|
|
130aa8cbcf
|
Add load pattern configuration guide to benchmarks (#26886)
Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com>
Signed-off-by: Matvei Pashkovskii <matvei.pashkovskii@amd.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-10-28 10:49:15 -07:00 |
|
|
|
e3d8186666
|
[compile] Add fallback path to AOT compile when serialization fails. (#27350)
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-10-28 12:54:26 -04:00 |
|
|
|
f5710ef02a
|
[Misc] Make LayerBlockType a Literal instead of Enum (#27658)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-28 16:23:35 +00:00 |
|
|
|
a8c02fb5bf
|
[Bugfix][CI] Fix v1 attention backend tests and add CI coverage (#26597)
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
Signed-off-by: Mohammad Miadh Angkad <mangkad.bsdsba2027@aim.edu>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
|
2025-10-28 11:42:05 -04:00 |
|
|
|
02af36df36
|
[Bugfix] Fix allocation & free logic of SingleWriterShmRingBuffer (#27117)
Signed-off-by: Kero Liang <kerorek@outlook.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: donglu <donglu@cohere.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2025-10-28 15:01:24 +00:00 |
|
|
|
e88bdd60d9
|
[FLA] Introduce Kimi Delta Attention(KDA) to VLLM (#27654)
Signed-off-by: lizhiyuan <lizhiyuan@moonshot.cn>
|
2025-10-28 22:56:28 +08:00 |
|
|
|
05e034f085
|
[nit]: Fix import for the lmcache integration (#27600)
Signed-off-by: Samuel Shen <slshen@uchicago.edu>
Co-authored-by: Samuel Shen <slshen@uchicago.edu>
|
2025-10-28 14:40:55 +00:00 |
|
|
|
936643a868
|
[BugFix] Also consider RAY_EXPERIMENTAL_NOSET_* when storing compilation cache (#27294)
Signed-off-by: Hollow Man <hollowman@opensuse.org>
|
2025-10-28 10:22:28 -04:00 |
|
|
|
b186149e8e
|
[Bugfix][Frontend] validate arg priority in frontend LLM class before add request (#27596)
Signed-off-by: Junpu Fan <junpufan@gmail.com>
|
2025-10-28 14:02:43 +00:00 |
|
|
|
2abbd351ef
|
[Core] Enable async scheduling for external_launcher mode (#27394)
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2025-10-28 13:52:47 +00:00 |
|
|
|
446912d1cb
|
fix: allow HuggingFace standard chat template params via **kwargs (#27622)
Signed-off-by: wangln19 <wanglinian@dev.wanglinian.msh-dev.svc.cluster.local>
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Co-authored-by: wangln19 <wanglinian@dev.wanglinian.msh-dev.svc.cluster.local>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2025-10-28 21:12:34 +08:00 |
|
|
|
a00d6254e9
|
[compile] Disable dynamo guards check for AOT compilation. (#27288)
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-10-28 12:58:12 +00:00 |
|
|
|
05181cc57f
|
[Hybrid] Add mamba_block_size to Engine Args (#27289)
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
|
2025-10-28 12:54:24 +00:00 |
|
|
|
259504e147
|
[compile] Add enable_prompt_embeds to compile hash. (#27285)
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-10-28 20:46:03 +08:00 |
|
|
|
0484b64248
|
[Bug] Fix shape issue for eplb expert weights (#27589)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-10-28 20:44:05 +08:00 |
|
|
|
f58d9b6404
|
[Misc] Separate out utils.counter and move utils.Device to engine (#27588)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-28 12:20:46 +00:00 |
|
|
|
44b5ce956d
|
[Bugfix] In LongRoPE, decide short vs long based on max_model_len (#27431)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2025-10-28 12:00:56 +00:00 |
|
|
|
7a865f2325
|
[V0 Deprecation] Remove vestigial V0 logits_processors.py file (#27601)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-10-28 19:17:45 +08:00 |
|
|
|
2fa90bda27
|
Fix a robust parsing issue in KimiK2ToolParser that causes IndexError (#27565)
Signed-off-by: wangln19 <wanglinian@dev.wanglinian.msh-dev.svc.cluster.local>
Co-authored-by: wangln19 <wanglinian@dev.wanglinian.msh-dev.svc.cluster.local>
|
2025-10-28 11:11:50 +00:00 |
|
|
|
0291fbf65c
|
[CI/Build] Fix amd model executor test (#27612)
Signed-off-by: zhewenli <zhewenli@meta.com>
|
2025-10-28 08:58:11 +00:00 |
|
|
|
b46e4a06f1
|
[Core][Bookkeeping Optimization] Update against numpy view of is_token_ids tensor (#27618)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
2025-10-28 08:13:10 +00:00 |
|
|
|
d34f5fe939
|
[Bugfix][CPU] Fallback oneDNN linear to torch linear to fix half gemm support on legecy platforms (#27526)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2025-10-27 23:25:44 -07:00 |
|
|
|
bdb01a38fe
|
[Hardware][AMD][Model] Triton MoE tuning configs for GLM-4.6 for MI300X (#27323)
Signed-off-by: minatoaquaMK2 <jiacheng.yue@foxmail.com>
|
2025-10-27 22:58:06 -07:00 |
|
|
|
5b3c35a68e
|
[ROCm] [Doc] Update ROCm installation docs (#27327)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
|
2025-10-28 13:00:50 +08:00 |
|
|
|
61fbfe5274
|
[Bugfix] fixed inconsistent finish_reason handling between V0 and V1 engines (#27555)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2025-10-28 02:18:08 +00:00 |
|
|
|
255e34ca50
|
[Stability fix] turn off HMA allocator when connector is set (#27592)
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
|
2025-10-27 18:32:23 -07:00 |
|
|
|
a8d2e326ec
|
[Bugfix][CI] Fix config resolving logic with remote models (#27610)
|
2025-10-28 00:48:32 +00:00 |
|
|
|
53a56e658b
|
[gpt-oss][2/N] Support input_messages in responsesRequest (#26962)
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
|
2025-10-27 23:15:49 +00:00 |
|
|
|
69f064062b
|
Code quality improvements: version update, type annotation enhancement, and enum usage simplification (#27581)
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
|
2025-10-27 17:50:22 +00:00 |
|
|
|
921e78f4bb
|
[ROCm] Update AITER branch for ROCm base docker (#27586)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
|
2025-10-27 17:22:33 +00:00 |
|
|
|
6ebffafbb6
|
[Misc] Clean up more utils (#27567)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-27 15:30:38 +00:00 |
|
|
|
3b96f85c36
|
[Chore]: Stream tokens vs characters in tool call parser tests (#26513)
Signed-off-by: Ben Browning <bbrownin@redhat.com>
|
2025-10-27 23:06:25 +08:00 |
|
|
|
23ad820553
|
fixing mm placeholder replacement issue with gemma3 (#27538)
Signed-off-by: tingtingtang1992 <streamttt@gmail.com>
|
2025-10-27 14:34:01 +00:00 |
|
|
|
5d3be3ba4c
|
[Bugfix][LoRA][FusedMoE] Select MxFP4 Backend based on LoRA Enablement (#27487)
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
|
2025-10-27 07:32:50 -07:00 |
|
|
|
4f882be4a0
|
[Model] Siglip2 Model Support (#27566)
Signed-off-by: piood <2477084691@qq.com>
|
2025-10-27 06:57:37 -07:00 |
|
|
|
9273754222
|
[Hybrid] Added supports_mamba_prefix_caching Protocol (#27339)
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
|
2025-10-27 13:05:20 +00:00 |
|
|
|
f4e8154076
|
[Kernel] Enable moe LoRA kernel support FP16 (#27468)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2025-10-27 19:48:37 +08:00 |
|
|
|
a663f6ae64
|
[cpu][perf] Fix low CPU utilization with VLLM_CPU_OMP_THREADS_BIND on AArch64 (#27415)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
|
2025-10-27 11:14:55 +00:00 |
|
|
|
a4fc21895e
|
[Bugfix] Fixed when return_token_ids=False, the first event still contains prompt_token_ids. (#27561)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2025-10-27 11:06:43 +00:00 |
|
|
|
a3e8611da5
|
[Bugfix] Limit the default value of max_model_len when it is not specified by users (#27556)
Signed-off-by: shen-shanshan <467638484@qq.com>
|
2025-10-27 10:16:20 +00:00 |
|
|
|
7c2bdb83dc
|
[Misc] Clean up utils (#27552)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-27 09:05:40 +00:00 |
|
|
|
9932ed6a83
|
[Kernel] Adding split_K implementation for fused_moe_lora (#27291)
Signed-off-by: Danielle Robinson <dmmaddix@amazon.com>
Signed-off-by: Danielle Robinson <dcmaddix@gmail.com>
Co-authored-by: Danielle Robinson <dmmaddix@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2025-10-27 02:05:24 -07:00 |
|
|
|
2d631d28c6
|
[Doc] Slight improvement to M2 and beyond (#27554)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2025-10-27 09:02:10 +00:00 |
|
|
|
b368382964
|
[Model] Deprecate merge_by_field_config=False (#27551)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-27 16:43:00 +08:00 |
|
|
|
a806c14cc7
|
[Performance][LoRA] add context varying params to 'do_not_specialize' in fused moe lora (#27445)
Signed-off-by: gnovack <gnovack@amazon.com>
|
2025-10-27 06:31:55 +00:00 |
|
|
|
181bf5bbde
|
[Docs] reemove the incorrect enable_reasoning parameter (#27550)
Signed-off-by: zxw <1020938856@qq.com>
|
2025-10-26 23:17:19 -07:00 |
|
|
|
cbd5e07a51
|
[Model] Use merge_by_field_config for MM models (Qwen series) (#27546)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-27 05:38:05 +00:00 |
|
|
|
63b22e0dbb
|
[Model][Bugfix] fix ernie45 moe 300B SharedFusedMoE output tuple (#27316)
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
|
2025-10-26 20:53:31 -07:00 |
|
