youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,797 Commits 157 Branches 93 Tags
4adc66f64d56338489d00d94de6e13d95741c4be
Commit Graph

84 Commits

Author SHA1 Message Date
Ilya Markov
37a7d5d74a [Misc] Refactor AllReduceFusionPass. Remove parameter (#20918) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-07-15 06:57:40 +00:00
Richard Zou
ba8c300018 [BugFix] VLLM_DISABLE_COMPILE_CACHE=1 should disable all reads and writes from the cache (#20942) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-07-15 01:26:18 +00:00
Ilya Markov
fc0f41d10a Integration SM100 FlashInfer fused allreduce RMSNorm (#20691) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-07-11 18:58:15 -07:00
Luka Govedič
762be26a8e [Bugfix] Upgrade depyf to 0.19 and streamline custom pass logging (#20777) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
 2025-07-11 00:15:22 -07:00
Luka Govedič
31d5c1797f [Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 04:56:28 +00:00
Jee Jee Li
1caca5a589 [Misc] Add SPDX-FileCopyrightText (#20428) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-04 07:40:42 +00:00
Boyuan Feng
c01d1c5aba use .dev for version comparison with pytorch nightly release (#20031) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-06-24 21:52:16 +00:00
cascade
e6327c9b3e [Feature] Support sequence parallelism for static fp8 quantization (#19181) 
Signed-off-by: cascade812 <cascade812@outlook.com>
 2025-06-23 16:09:02 -04:00
Maximilien de Bayser
799397ee4f Support embedding models in V1 (#16188) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-18 21:36:33 -07:00
Richard Zou
ed33349738 [BugFix] Fix use_cudagraph=False (#19612) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-06-19 08:23:12 +08:00
Charlie Fu
a44b1c951d [Feature][ROCm] Add full graph capture support for TritonAttentionBackend (#19158) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-06-17 17:03:06 -04:00
Luka Govedič
3597b06a4f [CUDA] Enable full cudagraph for FlashMLA (#18581) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-13 18:12:26 +00:00
Luka Govedič
f98548b9da [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
 2025-06-12 08:31:04 -07:00
Ning Xie
2f1c19b245 [CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
Richard Zou
77f0d465d0 [BugFix] Allow use_cudagraph to work with dynamic VLLM_USE_V1 (#19390) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-06-11 07:54:41 +08:00
Richard Zou
3a4d417707 [Misc] Cleanup compilation tests (#19343) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-06-09 15:05:44 +08:00
Richard Zou
3d64d366e0 [Misc] Change tests/compile to use VLLM_V1 by default (#19302) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-06-08 16:06:48 +08:00
Richard Zou
eaa2e51088 [Bugfix] Re-enable use_cudagraph in vLLM v1 (#19299) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-06-08 08:56:12 +08:00
Woosuk Kwon
b124e1085b [Bugfix] Fix FA3 full cuda graph correctness (#19106) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-03 23:10:15 -07:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Richard Zou
26b4fa45be Add ability to use CUDAGraphs with use_inductor=False (#17345) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-05-29 10:16:52 +08:00
cascade
71ea614d4a [Feature]Add async tensor parallelism using compilation pass (#17882) 
Signed-off-by: cascade812 <cascade812@outlook.com>
 2025-05-23 01:03:34 -07:00
Charlie Fu
7b2f28deba [AMD][torch.compile] Enable silu+fp8_quant fusion for rocm (#18082) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-05-13 22:13:56 -07:00
Aaron Pham
cb528d0585 [Fix] check to make sure processor has chat templates (#18047) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-05-13 03:04:10 -07:00
Harry Mellor
4b2ed7926a Improve configs - the rest! (#17562) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-09 15:18:44 -07:00
Chanh Nguyen
7ea2adb802 [Core] Support full cuda graph in v1 (#16072) 
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
 2025-05-07 22:30:15 -07:00
Richard Zou
b90b0852e9 [easy] Print number of needed GPUs in skip message (#17594) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-05-02 15:27:43 -07:00
Sage Moore
460a2b1100 [torch.compile] Add torch inductor pass for fusing silu_and_mul with subsequent scaled_fp8_quant operations (#10867) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-05-01 07:59:28 -07:00
Harry Mellor
b6dd32aa07 Make name of compressed-tensors quant method consistent across vLLM (#17255) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:28:13 +00:00
cascade
690fe019f0 [Feature] support sequence parallelism using compilation pass (#16155) 
Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-04-27 06:29:35 -07:00
Luka Govedič
9cdde47289 [BugFix] Fix fusion test and add them to CI (#16287) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-04-08 23:46:45 -07:00
Charlie Fu
e85829450d [Feature][ROCm]Enable fusion pass for torch.compile on ROCm (#15050) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-03-31 04:42:18 -07:00
Luka Govedič
04437e313d [Bugfix] [torch.compile] Add Dynamo metrics context during compilation (#15639) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-28 14:01:09 -06:00
Matthew Vine
7a6d45bc8a Support FIPS enabled machines with MD5 hashing (#15299) 
Signed-off-by: Matthew Vine <32849887+MattTheCuber@users.noreply.github.com>
 2025-03-26 20:19:46 -04:00
Luka Govedič
f622dbcf39 [Fix] [torch.compile] Improve UUID system for custom passes (#15249) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-24 01:54:07 +00:00
Jovan Sardinha
70e500cad9 Fix broken tests (#14713) 
Signed-off-by: JovanSardinha <jovan.sardinha@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-03-20 02:06:49 +00:00
Cyrus Leung
f690372b68 [Core] Update dtype detection and defaults (#14858) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-19 13:49:33 +08:00
vllmellm
2bb0e1a799 [Bugfix][ROCm] running new process using spawn method for rocm in tests. (#14810) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-17 11:33:35 +00:00
Sibi
a73e183e36 [Misc] Replace os environ to monkeypatch in test suite (#14516) 
Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-16 20:35:57 -07:00
Robert Shaw
d4d93db2c5 [V1] V1 Enablement Oracle (#13726) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-03-14 22:02:20 -07:00
Michael Goin
14f301b541 Update to torch==2.6.0 (#12721) 
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-14 16:58:30 -04:00
Luka Govedič
e1744502c2 [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-07 05:20:16 +00:00
Harry Mellor
cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Luka Govedič
bd56c983d6 [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-02-28 16:20:11 -07:00
Harry Mellor
f2b20fe491 Consolidate Llama model usage in tests (#13094) 2025-02-13 22:18:03 -08:00
youkaichao
09b95e36ab [torch.compile] PyTorch 2.6 and nightly compatibility (#12393) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-07 01:09:07 +08:00
Russell Bryant
e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Bowen Wang
2bc3fbba0c [FlashInfer] Upgrade to 0.2.0 (#11194) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-01-27 18:19:24 +00:00
youkaichao
3682e33f9f [v1] fix compilation cache (#11598) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-30 04:24:12 +00:00
Luka Govedič
30870b4f66 [torch.compile] Dynamic fp8 + rms_norm fusion (#10906) 
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2024-12-13 03:19:23 +00:00