youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5 Commits 157 Branches 93 Tags
5e644b912ec10ec0fac02ccdc0f9a4d8342d8f4c
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Woosuk Kwon 5e644b912e Add utils
2023-02-09 11:26:50 +00:00
cacheflow
Add utils
2023-02-09 11:26:50 +00:00
README.md
Initial commit
2023-02-09 11:24:15 +00:00

README.md

CacheFlow

Description
A high-throughput and memory-efficient inference and serving engine for LLMs
amdblackwellcudadeepseekdeepseek-v3gptgpt-ossinferencekimillamallmllm-servingmodel-servingmoeopenaipytorchqwenqwen3tputransformer
Readme Apache-2.0 687 MiB
Languages
Python 86.1%
Cuda 8.1%
C++ 4.2%
Shell 0.7%
C 0.4%
Other 0.4%