youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor attention kernels (#53)

Browse Source
This commit is contained in:
Woosuk Kwon
2023-05-03 13:40:13 -07:00
committed by GitHub
parent 27f1410d06
commit 436e523bf1
14 changed files with 1253 additions and 2569 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
setup.py
View File

@ -18,7 +18,7 @@ ext_modules.append(cache_extension)
# Attention kernels.
attention_extension = cpp_extension.CUDAExtension(
name='cacheflow.attention_ops',
sources=['csrc/attention.cpp', 'csrc/attention_kernels.cu'],
sources=['csrc/attention.cpp', 'csrc/attention/attention_kernels.cu'],
extra_compile_args={'cxx': CXX_FLAGS, 'nvcc': NVCC_FLAGS},
)
ext_modules.append(attention_extension)