Antoni Baum bfdb1ba5c3 [Core] Improve detokenization performance for prefill (#3469) ...

Co-authored-by: MeloYang <meloyang05@gmail.com>

2024-03-22 13:44:12 -07:00

..

async_engine

Asynchronous tokenization (#2879)

2024-03-15 23:37:01 +00:00

basic_correctness

[1/n][Chunked Prefill] Refactor input query shapes (#3236)

2024-03-20 14:46:05 -07:00

core

Dynamic scheduler delay to improve ITL performance (#3279)

2024-03-22 12:28:14 -07:00

distributed

[Test] Add basic correctness test (#2908)

2024-02-18 16:44:50 -08:00

engine

Asynchronous tokenization (#2879)

2024-03-15 23:37:01 +00:00

entrypoints

Support arbitrary json_object in OpenAI and Context Free Grammar (#3211)

2024-03-16 13:35:27 -07:00

kernels

[1/n] Triton sampling kernel (#3186)

2024-03-20 14:45:08 -07:00

lora

[Hardware][Neuron] Refactor neuron support (#3471)

2024-03-22 01:22:17 +00:00

metrics

Re-enable the 80 char line width limit (#3305)

2024-03-10 19:49:14 -07:00

models

[Bugfix][Model] Fix Qwen2 (#3554)

2024-03-22 00:18:58 +00:00

prefix_caching

[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)

2024-03-20 00:11:11 -07:00

prompts

[BugFix] Fix input positions for long context with sliding window (#2088)

2023-12-13 12:28:13 -08:00

samplers

Migrate logits computation and gather to model_runner (#3233)

2024-03-20 23:25:01 +00:00

spec_decode

[Hardware][Neuron] Refactor neuron support (#3471)

2024-03-22 01:22:17 +00:00

tokenization

[Core] Improve detokenization performance for prefill (#3469)

2024-03-22 13:44:12 -07:00

worker

[Hardware][Neuron] Refactor neuron support (#3471)

2024-03-22 01:22:17 +00:00

__init__.py

[Small] Formatter only checks lints in changed files (#1528)

2023-10-31 15:39:38 -07:00

conftest.py

Asynchronous tokenization (#2879)

2024-03-15 23:37:01 +00:00

test_cache_block_hashing.py

[CI/Build] Fix Bad Import In Test (#3473)

2024-03-18 20:28:00 +00:00

test_config.py

Fix assertion failure in Qwen 1.5 with prefix caching enabled (#3373)

2024-03-14 13:56:57 -07:00

test_logits_processor.py

Migrate logits computation and gather to model_runner (#3233)

2024-03-20 23:25:01 +00:00

test_regression.py

[BugFix] Fix GC bug for LLM class (#2882)

2024-02-14 22:17:44 -08:00

test_sampling_params.py

[Bugfix] fix crash if max_tokens=None (#2570)

2024-01-23 22:38:55 -08:00

test_sequence.py

[Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling (#3103)

2024-03-08 23:32:46 -08:00