youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,884 Commits 157 Branches 93 Tags
main
Commit Graph

109 Commits

Author SHA1 Message Date
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor
b893d661b1 Fix per file ruff ignores related to simplification (#26259) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 20:31:53 +00:00
Harry Mellor
4e256cadc2 Remove all references to yapf as it's no longer used (#26251) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 09:18:11 -07:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Reza Barazesh
bc546f76a1 [CI] Move applicable tests to CPU (#24080) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-30 14:45:20 +01:00
Russell Bryant
3958b96bf5 Add option to restrict media domains (#25783) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Chenheli Hua <huachenheli@outlook.com>
 2025-09-27 01:23:52 +00:00
Cyrus Leung
6b0fcbbf43 [Misc] Simplify test_argsort_mm_positions (#25690) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 18:23:01 +00:00
Cyrus Leung
6488f3481b [Misc]] Move processing context to multimodal directory (#25548) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-24 08:15:00 +00:00
Cyrus Leung
babad6e5dd [Misc] Move DP for ViT code inside model executor dir (#25459) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-23 09:20:52 +00:00
Deboleina
5aeb925452 Multimodal - audio tests (#25285) 
Signed-off-by: Debolina Roy <debroy@redhat.com>
 2025-09-22 07:07:11 +08:00
Harry Mellor
c4afdb69cc Move MultiModalConfig from config/__init__.py to config/multimodal.py (#24659) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-15 17:43:16 +00:00
dongluw
a5b84f1cbf [Core] Shared memory based object store for Multimodal data caching and IPC (#20452) 
Signed-off-by: donglu <donglu@cohere.com>
 2025-09-12 07:54:17 -07:00
Isotr0py
bcbe2a4d9e [VLM] Optimize GLM4.5-V-style video processing to only decode necessary frames (#24161) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-11 09:44:34 -07:00
dsinghvi
70549c1245 [CI/Build] Serve images used by multimodal tests through local HTTP Server (#23907) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-03 16:13:11 +08:00
Didier Durand
d7e1e59972 [Doc]: fix typos in Python comments (#24093) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-02 21:05:45 -07:00
WeiQing Chen
a0e0efd6bd [Model] Support DP for ViT on Kimi-VL-A3B-Thinking-2506 (#23817) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Signed-off-by: LJH-LBJ <98734602+LJH-LBJ@users.noreply.github.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: LJH-LBJ <98734602+LJH-LBJ@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-09-01 16:56:56 +00:00
Cyrus Leung
69244e67e6 [Core] Use key-only cache for BaseMultiModalProcessor (#23018) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-27 14:19:13 +08:00
Cyrus Leung
9715f7bb0f [Bugfix] Fix incorrect original shape in hashing (#23672) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-08-26 19:01:25 +00:00
Cyrus Leung
6879cd80ae [Refactor] Pass tokenizer explicitly instead of binding to prompt update (#23542) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-25 06:31:57 -07:00
Cyrus Leung
712d0f88d8 [Refactor] Dynamic target and content for prompt updates (#23411) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-24 23:39:58 -07:00
TJian
1298c67795 [FEAT] [Performance] Enable DP for ViT in Qwen2.5VL (#22742) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-08-19 15:25:57 +00:00
Cyrus Leung
27e8d1ea3e [Refactor] Define MultiModalKwargsItems separate from MultiModalKwargs (#23053) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-18 09:52:00 +00:00
Cyrus Leung
5c32143b9d [Refactor] Defer tensor data construction in MultiModalKwargs (#23030) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-16 21:05:50 -07:00
Staszek Paśko
22341b996e Improve multimodal hasher performance for re-used Image prompts (#22825) 
Signed-off-by: Staszek Pasko <staszek@gmail.com>
 2025-08-15 12:32:56 +00:00
Sayandip Dutta
aa300c438d [Bugfix] Unquote file uri before reading image (#22912) 
Signed-off-by: Sayandip Dutta <sayandip199309@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-08-15 09:28:00 +00:00
Cyrus Leung
19b927e52d [Core] Use individual MM items in P0/P1 cache and model runner (#22570) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-13 07:18:07 -07:00
Roger Wang
08b751ba74 Implicit language-model-only mode via limit-mm-per-prompt (#22299) 
Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Signed-off-by: Junhao Li <junhao@ubicloud.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Signed-off-by: Linkun <github@lkchen.net>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
Co-authored-by: Zhiyu <zhiyuc@nvidia.com>
Co-authored-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
Co-authored-by: Junhao Li <streaver91@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Yuxuan Zhang <2448370773@qq.com>
Co-authored-by: ZiTian Zhao <zitian.zhao@tencentmusic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Hong Hanh <hanh.usth@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: lkchen <github@lkchen.net>
 2025-08-08 22:21:40 -07:00
Cyrus Leung
766bc8162c [Core] Store only the keys for multi-modal data in P0 (#22198) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-07 01:45:04 -07:00
Cyrus Leung
f5d0f4784f [Frontend] Improve error message for too many mm items (#22114) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-02 02:20:38 -07:00
Gamhang
0a6d305e0f feat(multimodal): Add customizable background color for RGBA to RGB conversion (#22052) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
Co-authored-by: Jinheng Li <ahengljh@gmail.com>
 2025-08-01 06:07:33 -07:00
Cyrus Leung
82de9b9d46 [Misc] Automatically resolve HF processor init kwargs (#22005) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-31 22:44:10 -07:00
Cyrus Leung
86ae693f20 [Deprecation][2/N] Replace --task with --runner and --convert (#21470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-27 19:42:40 -07:00
Isotr0py
18e519ec86 [Bugfix] Fix ndarray video color from VideoAsset (#21064) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-07-19 02:17:16 -07:00
Isotr0py
147afb448b [Bugfix] Replace unavailable video url in multimodal test (#20854) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-07-12 05:25:39 +00:00
Chenheli Hua
b616f6a53d [Misc] Small: Fix video loader return type annotations. (#20389) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-07-03 03:10:39 +00:00
Chenheli Hua
2e7cbf2d7d [Frontend] Support configurable mm placeholder strings & flexible video sampling policies via CLI flags. (#20105) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-07-01 23:34:03 -07:00
Yuxuan Zhang
ed70f3c64f Add GLM4.1V model (Draft) (#19331) 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-01 12:48:26 +00:00
Kyle Sayers
d8cf819a9a [Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-06-30 17:26:49 +00:00
Lukas Geiger
3b523e38d9 [Core] Do not copy array during hashing (#19484) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-06-18 15:36:55 -07:00
Siqi Yan
f168b85725 Unit Test for run_dp_sharded_vision_model (#19103) 
Signed-off-by: Siqi Yan <siqi@meta.com>
Co-authored-by: Siqi Yan <siqi@meta.com>
 2025-06-06 16:24:02 +08:00
Guillaume Calmettes
9bc8bb07cf [Bugfix] properly catch PIL-related errors for vision models when incorrect data urls are provided (#19202) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-06-05 12:59:28 +00:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Chenheli Hua
04eb88dc80 Re-submit: Fix: Proper RGBA -> RGB conversion for PIL images. (#18569) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-05-23 01:59:18 +00:00
Chenheli Hua
4f07a64075 Support custom implementations of VideoLoader backends. (#18091) 2025-05-15 13:26:49 +08:00
Isotr0py
be8ff88e66 [Bugfix] Fix Video IO error for short video (#17791) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-07 15:36:06 +00:00
Cyrus Leung
99404f53c7 [Security] Fix image hash collision (#17378) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-02 08:36:39 -04:00
Cyrus Leung
d9fc8cd9da [V1] Enable multi-input by default (#15799) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-12 08:52:39 +00:00
Roger Wang
f2ebb6f541 [V1] Scatter and gather placeholders in the model runner (#16076) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
 2025-04-08 10:43:41 +08:00
Roger Wang
af51d80fa1 Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) 2025-04-04 14:50:57 -07:00
Cyrus Leung
f5722a5052 [V1] Scatter and gather placeholders in the model runner (#15712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-04 21:26:44 +00:00