|
|
f42b4c27d8
|
Include argmax to jit
|
2024-04-24 08:56:45 +00:00 |
|
|
|
620e7646d3
|
Fix cache write
|
2024-04-24 08:56:30 +00:00 |
|
|
|
d5fb1c20c1
|
Fix JAX jit OOM
|
2024-04-24 07:52:56 +00:00 |
|
|
|
092e3d6d6d
|
Remove hardcoded path
|
2024-04-19 08:18:10 +00:00 |
|
|
|
84284302d8
|
Minor
|
2024-04-19 08:08:25 +00:00 |
|
|
|
743695f586
|
Fix write_to_kv_cache
|
2024-04-19 07:51:54 +00:00 |
|
|
|
62b870fa07
|
Use FlashAttention kernel
|
2024-04-17 20:24:45 +00:00 |
|
|
|
7e3a230c38
|
Fix paged_attn
|
2024-04-17 20:06:26 +00:00 |
|
|
|
186c88c497
|
explictly return new_kv_caches
|
2024-04-17 18:42:34 +00:00 |
|
|
|
ef762cb110
|
Write kV
|
2024-04-17 18:21:39 +00:00 |
|
|
|
756c4e78d3
|
Add write_to_cache ops
|
2024-04-17 18:20:55 +00:00 |
|
|
|
4880de35d2
|
Add attn_mask
|
2024-04-17 18:12:20 +00:00 |
|
|
|
0fb07c08d0
|
Minor
|
2024-04-17 18:08:33 +00:00 |
|
|
|
e4377dd698
|
Add model runner
|
2024-04-17 18:04:54 +00:00 |
|
|
|
5cb213c85e
|
Add flash-attn op
|
2024-04-17 18:02:28 +00:00 |
|
|
|
25bbc21ef6
|
Minor
|
2024-04-17 18:02:16 +00:00 |
|
|
|
b25fcc06c2
|
Minor
|
2024-04-17 18:02:13 +00:00 |
|
|
|
6661c030c4
|
Add paged_attn op
|
2024-04-17 18:02:00 +00:00 |
|
|
|
8888d1c474
|
Fix logit indices
|
2024-04-17 18:01:43 +00:00 |
|
|
|
cedb67028a
|
Add gemma
|
2024-04-17 17:00:10 +00:00 |
|
|
|
91b47e3f2f
|
JAX-based TPU worker
|
2024-04-16 17:37:11 +00:00 |
|
|
|
6d62e4c6aa
|
Add torch to dependencies
|
2024-04-16 17:06:35 +00:00 |
|
|
|
de82e95787
|
Minor
|
2024-04-16 17:04:46 +00:00 |
|
|
|
b3b89cf755
|
Renew TPU executor
|
2024-04-16 09:42:15 +00:00 |
|
|
|
6692a30266
|
Minor
|
2024-04-16 09:41:53 +00:00 |
|
|
|
eb0a0466a9
|
Add JAX requirements
|
2024-04-16 08:05:54 +00:00 |
|
|
|
c59c1e7b2c
|
Remove
|
2024-04-16 08:05:36 +00:00 |
|
|
|
d4adf92beb
|
Merge branch 'main' into woosuk-tpu
|
2024-04-16 07:56:53 +00:00 |
|
|
|
05434764cd
|
LM Format Enforcer Guided Decoding Support (#3868)
Co-authored-by: Simon Mo <simon.mo@hey.com>
|
2024-04-16 05:54:57 +00:00 |
|
|
|
4e7ee664e2
|
[Core] Fix engine-use-ray broken (#4105)
|
2024-04-16 05:24:53 +00:00 |
|
|
|
37e84a403d
|
[Typing] Fix Sequence type GenericAlias only available after Python 3.9. (#4092)
|
2024-04-15 14:47:31 -07:00 |
|
|
|
4695397dcf
|
[Bugfix] Fix ray workers profiling with nsight (#4095)
|
2024-04-15 14:24:45 -07:00 |
|
|
|
d619ae2d19
|
[Doc] Add better clarity for tensorizer usage (#4090)
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
|
2024-04-15 13:28:25 -07:00 |
|
|
|
eb46fbfda2
|
[Core] Simplifications to executor classes (#4071)
|
2024-04-15 13:05:09 -07:00 |
|
|
|
0003e9154b
|
[Misc][Minor] Fix CPU block num log in CPUExecutor. (#4088)
|
2024-04-15 08:35:55 -07:00 |
|
|
|
e11e200736
|
[Bugfix] Fix filelock version requirement (#4075)
|
2024-04-14 21:50:08 -07:00 |
|
|
|
8db1bf32f8
|
[Misc] Upgrade triton to 2.2.0 (#4061)
|
2024-04-14 17:43:54 -07:00 |
|
|
|
aceb17cf2d
|
[Docs] document that mixtral 8x22b is supported (#4073)
|
2024-04-14 14:35:55 -07:00 |
|
|
|
563c54f760
|
[BugFix] Fix tensorizer extra in setup.py (#4072)
|
2024-04-14 14:12:42 -07:00 |
|
|
|
2cd6b4f362
|
[Core] avoid too many cuda context by caching p2p test (#4021)
|
2024-04-13 23:40:21 -07:00 |
|
|
|
711a000255
|
[Frontend] [Core] feat: Add model loading using tensorizer (#3476)
|
2024-04-13 17:13:01 -07:00 |
|
|
|
989ae2538d
|
[Kernel] Add punica dimension for Baichuan-13B (#4053)
|
2024-04-13 07:55:05 -07:00 |
|
|
|
0a430b4ae2
|
[Bugfix] fix_small_bug_in_neuron_executor (#4051)
|
2024-04-13 07:54:03 -07:00 |
|
|
|
ec8e3c695f
|
[Bugfix] fix_log_time_in_metrics (#4050)
|
2024-04-13 07:52:36 -07:00 |
|
|
|
98afde19fc
|
[Core][Distributed] improve logging for init dist (#4042)
|
2024-04-13 07:12:53 -07:00 |
|
|
|
5c2e66e487
|
[Bugfix] More type hint fixes for py 3.8 (#4039)
|
2024-04-12 21:07:04 -07:00 |
|
|
|
546e721168
|
[CI/Test] expand ruff and yapf for all supported python version (#4037)
|
2024-04-13 01:43:37 +00:00 |
|
|
|
b8aacac31a
|
[Bugfix] Fix LoRA bug (#4032)
|
2024-04-12 16:56:37 -07:00 |
|
|
|
d04973ad54
|
Fix triton compilation issue (#3984)
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-04-12 16:41:26 -07:00 |
|
|
|
fbb9d9eef4
|
[Core] fix custom allreduce default value (#4040)
|
2024-04-12 16:40:39 -07:00 |
|
