|
|
1efce68605
|
[Bugfix] Use runner_type instead of task in GritLM (#11144)
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
|
2024-12-13 04:09:53 +00:00 |
|
|
|
30870b4f66
|
[torch.compile] Dynamic fp8 + rms_norm fusion (#10906)
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2024-12-13 03:19:23 +00:00 |
|
|
|
78ed8f57d8
|
[Misc][V1] Fix type in v1 prefix caching (#11151)
|
2024-12-13 00:57:40 +00:00 |
|
|
|
db6c264a1e
|
[Bugfix] Fix value unpack error of simple connector for KVCache transfer. (#11058)
Signed-off-by: ShangmingCai <csmthu@gmail.com>
|
2024-12-12 21:19:17 +00:00 |
|
|
|
9f3974a319
|
Fix logging of the vLLM Config (#11143)
|
2024-12-12 12:05:57 -08:00 |
|
|
|
2c97eca1ff
|
[Misc] Validate grammar and fail early (#11119)
|
2024-12-12 18:34:26 +00:00 |
|
|
|
5d712571af
|
[Bugfix] Quick fix to make Pixtral-HF load correctly again after 39e227c7ae. (#11024)
|
2024-12-12 18:09:20 +00:00 |
|
|
|
d4d5291cc2
|
fix(docs): typo in helm install instructions (#11141)
Signed-off-by: Ramon Ziai <ramon.ziai@bettermarks.com>
|
2024-12-12 17:36:32 +00:00 |
|
|
|
4816d20aa4
|
[V1] Fix torch profiling for offline inference (#11125)
Signed-off-by: Roger Wang <ywang@roblox.com>
|
2024-12-12 15:51:53 +00:00 |
|
|
|
85362f028c
|
[Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094)
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-12-12 09:25:16 +00:00 |
|
|
|
62de37a38e
|
[core][distributed] initialization from StatelessProcessGroup (#10986)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-12 09:04:19 +00:00 |
|
|
|
8195824206
|
[Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (#10565)
Signed-off-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
|
2024-12-12 08:09:28 +00:00 |
|
|
|
f092153fbe
|
[V1] Use more persistent buffers to optimize input preparation overheads (#11111)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-12-11 23:14:20 -08:00 |
|
|
|
1da8f0e1dd
|
[Model] Add support for embedding model GritLM (#10816)
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
|
2024-12-12 06:39:16 +00:00 |
|
|
|
ccede2b264
|
[Core] cleanup zmq ipc sockets on exit (#11115)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2024-12-11 19:12:24 -08:00 |
|
|
|
24a36d6d5f
|
Update link to LlamaStack remote vLLM guide in serving_with_llamastack.rst (#11112)
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
|
2024-12-12 02:39:21 +00:00 |
|
|
|
8fb26dac61
|
[Docs] Add media kit (#11121)
|
2024-12-11 17:33:11 -08:00 |
|
|
|
7439a8b5fc
|
[Bugfix] Multiple fixes to tool streaming with hermes and mistral (#10979)
Signed-off-by: cedonley <clayton@donley.io>
|
2024-12-12 01:10:12 +00:00 |
|
|
|
4e11683368
|
[V1] VLM preprocessor hashing (#11020)
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-12-12 00:55:30 +00:00 |
|
|
|
452a723bf2
|
[V1][Core] Remove should_shutdown to simplify core process termination (#11113)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2024-12-11 23:34:54 +00:00 |
|
|
|
d1e21a979b
|
[CI/Build] Split up VLM tests (#11083)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-12 06:18:16 +08:00 |
|
|
|
72ff3a9686
|
[core] Bump ray to use _overlap_gpu_communication in compiled graph tests (#10410)
Signed-off-by: Rui Qiao <ubuntu@ip-172-31-15-128.us-west-2.compute.internal>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Rui Qiao <ubuntu@ip-172-31-15-128.us-west-2.compute.internal>
|
2024-12-11 11:36:35 -08:00 |
|
|
|
66aaa7722d
|
[torch.compile] remove graph logging in ci (#11110)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-11 10:59:50 -08:00 |
|
|
|
d643c2aba1
|
[V1] Use input_ids as input for text-only models (#11032)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-12-11 10:49:23 -08:00 |
|
|
|
91642db952
|
[torch.compile] use depyf to dump torch.compile internals (#10972)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-11 10:43:05 -08:00 |
|
|
|
fd22220687
|
[Doc] Installed version of llmcompressor for int8/fp8 quantization (#11103)
Signed-off-by: Guangda Liu <bingps@users.noreply.github.com>
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
|
2024-12-11 15:43:24 +00:00 |
|
|
|
b2f775456e
|
[CI/Build] Enable prefix caching test for AMD (#11098)
Signed-off-by: Hissu Hyvarinen <hissu.hyvarinen@amd.com>
|
2024-12-11 15:23:37 +00:00 |
|
|
|
cad5c0a6ed
|
[Doc] Update docs to refer to pooling models (#11093)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-11 13:36:27 +00:00 |
|
|
|
8f10d5e393
|
[Misc] Split up pooling tasks (#10820)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-11 01:28:00 -08:00 |
|
|
|
40766ca1b8
|
[Bugfix]: Clamp -inf logprob values in prompt_logprobs (#11073)
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
|
2024-12-11 01:27:39 -08:00 |
|
|
|
2e32f5d28d
|
[Bugfix] Fix Idefics3 fails during multi-image inference (#11080)
Signed-off-by: B-201 <Joy25810@foxmail.com>
|
2024-12-11 01:27:07 -08:00 |
|
|
|
61b1d2f6ae
|
[Core] v1: Use atexit to handle engine core client shutdown (#11076)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2024-12-11 01:26:36 -08:00 |
|
|
|
9974fca047
|
[ci/build] Fix entrypoints test and pin outlines version (#11088)
|
2024-12-11 01:01:53 -08:00 |
|
|
|
3fb4b4f163
|
[ci/build] Fix AMD CI dependencies (#11087)
|
2024-12-11 00:39:53 -08:00 |
|
|
|
2e33fe4191
|
[CI/Build] Check transformers v4.47 (#10991)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-11 05:02:02 +00:00 |
|
|
|
e39400a4b6
|
Fix streaming for granite tool call when <|tool_call|> is present (#11069)
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
|
2024-12-11 04:51:40 +00:00 |
|
|
|
ffa48c9146
|
[Model] PP support for Mamba-like models (#10992)
Signed-off-by: mzusman <mor.zusmann@gmail.com>
|
2024-12-10 21:53:37 -05:00 |
|
|
|
d5c5154fcf
|
[Misc] LoRA + Chunked Prefill (#9057)
|
2024-12-11 10:09:20 +08:00 |
|
|
|
9a93973708
|
[Bugfix] Fix Mamba multistep (#11071)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2024-12-11 00:16:22 +00:00 |
|
|
|
134810b3d9
|
[V1][Bugfix] Always set enable_chunked_prefill = True for V1 (#11061)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-12-10 14:41:23 -08:00 |
|
|
|
75f89dc44c
|
[torch.compile] add a flag to track batchsize statistics (#11059)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-10 12:40:52 -08:00 |
|
|
|
e739194926
|
[Core] Update to outlines >= 0.1.8 (#10576)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2024-12-10 12:08:16 -08:00 |
|
|
|
250ee65d72
|
[BUG] Remove token param #10921 (#11022)
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
|
2024-12-10 17:38:15 +00:00 |
|
|
|
9b9cef3145
|
[Bugfix] Backport request id validation to v0 (#11036)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
|
2024-12-10 16:38:23 +00:00 |
|
|
|
d05f88679b
|
[Misc][LoRA] Add PEFTHelper for LoRA (#11003)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-12-10 11:12:01 +00:00 |
|
|
|
beb16b2c81
|
[Bugfix] Handle <|tool_call|> token in granite tool parser (#11039)
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
|
2024-12-10 10:27:11 +00:00 |
|
|
|
fe2e10c71b
|
Add example of helm chart for vllm deployment on k8s (#9199)
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
|
2024-12-10 09:19:27 +00:00 |
|
|
|
82c73fd510
|
[Bugfix] cuda error running llama 3.2 (#11047)
|
2024-12-10 07:41:11 +00:00 |
|
|
|
bfd610430c
|
Update README.md (#11034)
|
2024-12-09 23:08:10 -08:00 |
|
|
|
e35879c276
|
[Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig on PixtralHF. (#11043)
|
2024-12-10 14:54:22 +08:00 |
|
