Antoni Baum 426ec4ec67 [1/n] Triton sampling kernel (#3186) ...

Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

2024-03-20 14:45:08 -07:00

..

async_engine

Asynchronous tokenization (#2879)

2024-03-15 23:37:01 +00:00

basic_correctness

[Test] Add basic correctness test (#2908)

2024-02-18 16:44:50 -08:00

core

[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)

2024-03-20 00:11:11 -07:00

distributed

[Test] Add basic correctness test (#2908)

2024-02-18 16:44:50 -08:00

engine

Asynchronous tokenization (#2879)

2024-03-15 23:37:01 +00:00

entrypoints

Support arbitrary json_object in OpenAI and Context Free Grammar (#3211)

2024-03-16 13:35:27 -07:00

kernels

[1/n] Triton sampling kernel (#3186)

2024-03-20 14:45:08 -07:00

lora

Asynchronous tokenization (#2879)

2024-03-15 23:37:01 +00:00

metrics

Re-enable the 80 char line width limit (#3305)

2024-03-10 19:49:14 -07:00

models

Re-enable the 80 char line width limit (#3305)

2024-03-10 19:49:14 -07:00

prefix_caching

[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)

2024-03-20 00:11:11 -07:00

prompts

[BugFix] Fix input positions for long context with sliding window (#2088)

2023-12-13 12:28:13 -08:00

samplers

[1/n] Triton sampling kernel (#3186)

2024-03-20 14:45:08 -07:00

spec_decode

Re-enable the 80 char line width limit (#3305)

2024-03-10 19:49:14 -07:00

tokenization

Asynchronous tokenization (#2879)

2024-03-15 23:37:01 +00:00

worker

[Misc] Remove cache stream and cache events (#3461)

2024-03-20 00:38:53 -07:00

__init__.py

[Small] Formatter only checks lints in changed files (#1528)

2023-10-31 15:39:38 -07:00

conftest.py

Asynchronous tokenization (#2879)

2024-03-15 23:37:01 +00:00

test_cache_block_hashing.py

[CI/Build] Fix Bad Import In Test (#3473)

2024-03-18 20:28:00 +00:00

test_config.py

Fix assertion failure in Qwen 1.5 with prefix caching enabled (#3373)

2024-03-14 13:56:57 -07:00

test_regression.py

[BugFix] Fix GC bug for LLM class (#2882)

2024-02-14 22:17:44 -08:00

test_sampling_params.py

[Bugfix] fix crash if max_tokens=None (#2570)

2024-01-23 22:38:55 -08:00

test_sequence.py

[Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling (#3103)

2024-03-08 23:32:46 -08:00