youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3 Commits 157 Branches 93 Tags
6680129ba2d9c779050a18a97aeb57930a0b2f35
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Woosuk Kwon 6680129ba2 Add blocks
2023-02-09 11:26:21 +00:00
cacheflow
Add blocks
2023-02-09 11:26:21 +00:00
README.md
Initial commit
2023-02-09 11:24:15 +00:00

README.md

CacheFlow

Description
A high-throughput and memory-efficient inference and serving engine for LLMs
amdblackwellcudadeepseekdeepseek-v3gptgpt-ossinferencekimillamallmllm-servingmodel-servingmoeopenaipytorchqwenqwen3tputransformer
Readme Apache-2.0 687 MiB
Languages
Python 86.1%
Cuda 8.1%
C++ 4.2%
Shell 0.7%
C 0.4%
Other 0.4%