youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,376 Commits 157 Branches 93 Tags
415f19474dedc69934cab79cfb8f5bdc19e2ae0d
Commit Graph

644 Commits

Author SHA1 Message Date
Cody Yu
60bcef000e [Docs][V1] Prefix caching design (#12598) 
- Create v1 design document section in docs.
- Add prefix caching design doc.

@WoosukKwon @ywang96

---------

Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-01-31 12:30:46 -08:00
Cody Yu
847f883232 [Git] Automatically sign-off commits (#12595) 
It's very annoying when I forgot to add `-s` in `git commit` to
sign-off, because I then need to `git rebase HEAD~1 --signoff` and `git
push -f` to fix the DCO. This PR adds a hook to sign off commits
automatically when `-s` is missing to solve this problem. The only
change from the user side is now users have to install 2 hooks, so
instead of just

```
pre-commit install
```

Now we need to

```
pre-commit install --hook-type pre-commit --hook-type commit-msg
```

Note that even if users still only install the pre-commit hook, they
won't get any error in `git commit`. Just the sign-off hook won't run.

cc @hmellor @youkaichao

---------

Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-01-31 12:30:33 -08:00
Harry Mellor
e3f7ff65e7 Add favicon to docs (#12611) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-31 09:20:34 -08:00
Harry Mellor
a2769032ca Set ?device={device} when changing tab in installation guides (#12560) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-30 00:05:42 -08:00
Alphi
d93bf4da85 [Model] Refactoring of MiniCPM-V and add MiniCPM-o-2.6 support for vLLM (#12069) 
Signed-off-by: hzh <hezhihui_thu@163.com>
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
Signed-off-by: Chenguang Li <757486878@qq.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Shanshan Shen <467638484@qq.com>
Signed-off-by: elijah <f1renze.142857@gmail.com>
Signed-off-by: Yikun <yikunkero@gmail.com>
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: shaochangxu <85155497+shaochangxu@users.noreply.github.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: sixgod <evethwillbeok@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Rafael Vasquez <rafvasq21@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Akshat Tripathi <Akshat.tripathi6568@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Avshalom Manevich <12231371+avshalomman@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Yangcheng Li <liyangcheng.lyc@alibaba-inc.com>
Co-authored-by: Siyuan Li <94890248+liaoyanqing666@users.noreply.github.com>
Co-authored-by: Concurrensee <yida.wu@amd.com>
Co-authored-by: Chenguang Li <757486878@qq.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Alex Brooks <alex.brooks@ibm.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: elijah <30852919+e1ijah1@users.noreply.github.com>
Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
Co-authored-by: Steve Luo <36296769+SunflowerAries@users.noreply.github.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: TJian <tunjian1996@gmail.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: maang-h <55082429+maang-h@users.noreply.github.com>
Co-authored-by: Elfie Guo <164945471+elfiegg@users.noreply.github.com>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-01-29 09:24:59 +00:00
Harry Mellor
dd6a3a02cb [Doc] Convert docs to use colon fences (#12471) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-29 11:38:29 +08:00
Ce Gao
a7e3eba66f [Frontend] Support reasoning content for deepseek r1 (#12473) 
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
Co-authored-by: Rafael Vasquez <rafvasq21@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
 2025-01-29 11:38:08 +08:00
Jun Duan
925d2f1908 [Doc] Fix typo for x86 CPU installation (#12514) 
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
 2025-01-28 16:37:10 +00:00
Cyrus Leung
8f58a51358 [VLM] Merged multi-modal processor and V1 support for Qwen-VL (#12504) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-28 16:25:05 +00:00
Yuan Tang
582cf78798 [DOC] Add link to vLLM blog (#12460) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2025-01-27 03:46:19 +00:00
Kyle Mistele
0034b09ceb [Frontend] Rerank API (Jina- and Cohere-compatible API) (#12376) 
Signed-off-by: Kyle Mistele <kyle@mistele.com>
 2025-01-26 19:58:45 -07:00
Mohit Deopujari
9a0f3bdbe5 [Hardware][Gaudi][Doc] Add missing step in setup instructions (#12382) 2025-01-24 09:43:49 +00:00
Russell Bryant
c5cffcd0cd [Docs] Update spec decode + structured output in compat matrix (#12373) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-01-24 01:15:52 +00:00
Woosuk Kwon
682b55bc07 [Docs] Add meetup slides (#12345) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-01-23 14:10:03 -08:00
Isotr0py
2cbeedad09 [Docs] Document Phi-4 support (#12362) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-23 19:18:51 +00:00
Gregory Shtrasberg
e97f802b2d [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
 2025-01-23 18:04:03 +00:00
Cyrus Leung
d07efb31c5 [Doc] Troubleshooting errors during model inspection (#12351) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-23 22:46:58 +08:00
youkaichao
511627445e [doc] explain common errors around torch.compile (#12340) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-23 14:56:02 +08:00
Russell Bryant
7551a34032 [Docs] Document vulnerability disclosure process (#12326) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-01-23 03:44:09 +00:00
Michael Goin
01a55941f5 [Docs] Update FP8 KV Cache documentation (#12238) 
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-01-23 11:18:09 +08:00
Hongxia Yang
09ccc9c8f7 [Documentation][AMD] Add information about prebuilt ROCm vLLM docker for perf validation purpose (#12281) 
Signed-off-by: Hongxia Yang <hongxyan@amd.com>
 2025-01-22 07:49:22 +08:00
Cyrus Leung
96912550c8 [Misc] Rename MultiModalInputsV2 -> MultiModalInputs (#12244) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-21 07:31:19 +00:00
Gregory Shtrasberg
d4b62d4641 [AMD][Build] Porting dockerfiles from the ROCm/vllm fork (#11777) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-01-21 12:22:23 +08:00
Isotr0py
83609791d2 [Model] Add Qwen2 PRM model support (#12202) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-20 14:59:46 +08:00
Harry Mellor
3ea7b94523 Move linting to pre-commit (#11975) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-20 14:58:01 +08:00
Roger Wang
81763c58a0 [V1] Add V1 support of Qwen2-VL (#12128) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-19 19:52:13 +08:00
Isotr0py
02798ecabe [Model] Port deepseek-vl2 processor, remove dependency (#12169) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-18 13:59:39 +08:00
Hongxia Yang
c09503ddd6 [AMD][CI/Build][Bugfix] use pytorch stale wheel (#12172) 
Signed-off-by: hongxyan <hongxyan@amd.com>
 2025-01-18 11:15:53 +08:00
Yuan Tang
1475847a14 [Doc] Add instructions on using Podman when SELinux is active (#12136) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2025-01-17 04:45:36 +00:00
Isotr0py
62b06ba23d [Model] Add support for deepseek-vl2-tiny model (#12068) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-16 17:14:48 +00:00
Cyrus Leung
f8ef146f03 [Doc] Add documentation for specifying model architecture (#12105) 2025-01-16 15:53:43 +08:00
RunningLeon
97eb97b5a4 [Model]: Support internlm3 (#12037) 2025-01-15 11:35:17 +00:00
Kyle Sayers
3f9b7ab9f5 [Doc] Update examples to remove SparseAutoModelForCausalLM (#12062) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-01-15 06:36:01 +00:00
Harry Mellor
c9d6ff530b Explain where the engine args go when using Docker (#12041) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-14 16:05:50 +00:00
TJian
8a1f938e6f [Doc] Update Quantization Hardware Support Documentation (#12025) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-01-14 04:37:52 +00:00
Woosuk Kwon
1a401252b5 [Docs] Add Sky Computing Lab to project intro (#12019) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-01-13 17:24:36 -08:00
Harry Mellor
e8c23ff989 [Doc] Organise installation documentation into categories and tabs (#11935) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-13 12:27:36 +00:00
Roger Wang
cd8249903f [Doc][V1] Update model implementation guide for V1 support (#11998) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-01-13 11:58:54 +00:00
Akshat Tripathi
8bddb73512 [Hardware][CPU] Multi-LoRA implementation for the CPU backend (#11100) 
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-01-12 13:01:52 +00:00
Isotr0py
f967e51f38 [Model] Initialize support for Deepseek-VL2 models (#11578) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-01-12 00:17:24 -08:00
Rafael Vasquez
43f3d9e699 [CI/Build] Add markdown linter (#11857) 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
 2025-01-12 00:17:13 -08:00
Cyrus Leung
a991f7d508 [Doc] Basic guide for writing unit tests for new models (#11951) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-11 21:27:24 +08:00
Li, Jiang
aa1e77a19c [Hardware][CPU] Support MOE models on x86 CPU (#11831) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-01-10 11:07:58 -05:00
Harry Mellor
482cdc494e [Doc] Rename offline inference examples (#11927) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-10 23:50:29 +08:00
Cyrus Leung
12664ddda5 [Doc] [1/N] Initial guide for merged multi-modal processor (#11925) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-10 14:30:25 +00:00
Harry Mellor
d85c47d6ad Replace "online inference" with "online serving" (#11923) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-10 12:05:56 +00:00
Cyrus Leung
3de2b1eafb [Doc] Show default pooling method in a table (#11904) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-10 11:25:20 +08:00
Cyrus Leung
c3cf54dda4 [Doc][5/N] Move Community and API Reference to the bottom (#11896) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-01-10 03:10:12 +00:00
Charles Frye
36f5303578 [Docs] Add Modal to deployment frameworks (#11907) 2025-01-09 23:26:37 +00:00
Cyrus Leung
9a228348d2 [Misc] Provide correct Pixtral-HF chat template (#11891) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-09 10:19:37 -07:00