Michael Goin fbb5bd4cef [TPU] Add example for profiling TPU inference (#12531) ...

Signed-off-by: mgoin <mgoin@redhat.com>

2025-01-29 03:16:47 +00:00

..

openai

[Frontend] Support scores endpoint in run_batch (#12430)

2025-01-27 04:30:17 +00:00

profiling_tpu

[TPU] Add example for profiling TPU inference (#12531)

2025-01-29 03:16:47 +00:00

aqlm_example.py

[Doc] Move examples into categories (#11840)

2025-01-08 13:09:53 +00:00

arctic.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

audio_language.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

basic_with_model_default_sampling.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

basic.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

chat_with_tools.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

chat.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

classification.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

cli.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

cpu_offload.py

[Doc] Move examples into categories (#11840)

2025-01-08 13:09:53 +00:00

distributed.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

embedding.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

encoder_decoder.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

florence2_inference.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

gguf_inference.py

[Model] Support GGUF models newly added in transformers 4.46.0 (#9685)

2025-01-13 00:13:44 +00:00

llm_engine_example.py

[Doc] Move examples into categories (#11840)

2025-01-08 13:09:53 +00:00

lora_with_quantization_inference.py

[Doc] Move examples into categories (#11840)

2025-01-08 13:09:53 +00:00

mlpspeculator.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

multilora_inference.py

[Doc] Move examples into categories (#11840)

2025-01-08 13:09:53 +00:00

neuron_int8_quantization.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

neuron.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

pixtral.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

prefix_caching.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

profiling.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

rlhf.py

[core][bugfix] configure env var during import vllm (#12209)

2025-01-20 19:35:59 +08:00

save_sharded_state.py

[Doc] Move examples into categories (#11840)

2025-01-08 13:09:53 +00:00

scoring.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

simple_profiling.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

structured_outputs.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

torchrun_example.py

Support torchrun and SPMD-style offline inference (#12071)

2025-01-16 19:58:53 +08:00

tpu.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

vision_language_embedding.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00

vision_language_multi_image.py

[Model] Port deepseek-vl2 processor, remove dependency (#12169)

2025-01-18 13:59:39 +08:00

vision_language.py

[Model][Bugfix]: correct Aria model output (#12309)

2025-01-22 11:39:19 +00:00

whisper.py

[Doc] Rename offline inference examples (#11927)

2025-01-10 23:50:29 +08:00