youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
61a6905ab036fd00eafdb1b0ca130d5feccfe686
vllm/examples/online_serving
History
Chauncey 63fe3a700f [PD] let p2p nccl toy proxy handle /chat/completions (#21734) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2025-07-28 11:45:50 +00:00
..
chart-helm
[V0 deprecation] Remove V0 CPU/XPU/TPU backends (#20412)
2025-07-06 08:48:13 -07:00
disaggregated_serving
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
disaggregated_serving_p2p_nccl_xpyd
[PD] let p2p nccl toy proxy handle /chat/completions (#21734)
2025-07-28 11:45:50 +00:00
elastic_ep
Elastic Expert Parallel Initial Support (#20775)
2025-07-18 17:46:09 -07:00
opentelemetry
[Docs] Fix syntax highlighting of shell commands (#19870)
2025-06-23 17:59:09 +00:00
prometheus_grafana
[CI/Build][Doc] Clean up more docs that point to old bench scripts (#21667)
2025-07-27 04:02:12 +00:00
structured_outputs
[Misc] small update (#20462)
2025-07-03 20:33:44 -07:00
api_client.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
cohere_rerank_client.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
disaggregated_prefill.sh
[Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824)
2025-03-20 00:00:33 -07:00
gradio_openai_chatbot_webserver.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
gradio_webserver.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
jinaai_rerank_client.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
kv_events_subscriber.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
multi_instance_data_parallel.py
[Misc] Add SPDX-FileCopyrightText (#20428)
2025-07-04 07:40:42 +00:00
multi-node-serving.sh
[Docs] Improve documentation for multi-node service helper script (#20600)
2025-07-08 19:44:26 -07:00
openai_chat_completion_client_for_multimodal.py
[Deprecation][2/N] Replace --task with --runner and --convert (#21470)
2025-07-27 19:42:40 -07:00
openai_chat_completion_client_with_tools_required.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_chat_completion_client_with_tools_xlam_streaming.py
[Misc] Add SPDX-FileCopyrightText (#20428)
2025-07-04 07:40:42 +00:00
openai_chat_completion_client_with_tools_xlam.py
[Misc] Add SPDX-FileCopyrightText (#20428)
2025-07-04 07:40:42 +00:00
openai_chat_completion_client_with_tools.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_chat_completion_client.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_chat_completion_tool_calls_with_reasoning.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_chat_completion_with_reasoning_streaming.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_chat_completion_with_reasoning.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_chat_embedding_client_for_multimodal.py
[Deprecation][2/N] Replace --task with --runner and --convert (#21470)
2025-07-27 19:42:40 -07:00
openai_classification_client.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_completion_client.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_cross_encoder_score_for_multimodal.py
[Deprecation][2/N] Replace --task with --runner and --convert (#21470)
2025-07-27 19:42:40 -07:00
openai_cross_encoder_score.py
[Deprecation][2/N] Replace --task with --runner and --convert (#21470)
2025-07-27 19:42:40 -07:00
openai_embedding_client.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_embedding_matryoshka_fy.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
openai_pooling_client.py
[Deprecation][2/N] Replace --task with --runner and --convert (#21470)
2025-07-27 19:42:40 -07:00
openai_transcription_client.py
[Docs] Update transcriptions API to use openai client with stream=True (#20271)
2025-07-01 15:47:13 +00:00
openai_translation_client.py
[Frontend] Add /v1/audio/translations OpenAI API endpoint (#19615)
2025-06-25 17:54:14 +00:00
prompt_embed_inference_with_openai_client.py
[Deprecation][2/N] Replace --task with --runner and --convert (#21470)
2025-07-27 19:42:40 -07:00
ray_serve_deepseek.py
[Docs] Improve documentation for Deepseek R1 on Ray Serve LLM (#20601)
2025-07-08 02:09:06 -07:00
retrieval_augmented_generation_with_langchain.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
retrieval_augmented_generation_with_llamaindex.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00
run_cluster.sh
[Docs] Improve documentation for ray cluster launcher helper script (#20602)
2025-07-15 03:55:45 -07:00
sagemaker-entrypoint.sh
[Doc] Move examples into categories (#11840)
2025-01-08 13:09:53 +00:00
streamlit_openai_chatbot_webserver.py
[DOC] Add reasoning capability to vLLM streamlit code (#19557)
2025-06-16 07:09:12 -04:00
utils.py
[Misc] Add SPDX-FileCopyrightText (#19100)
2025-06-03 11:20:17 -07:00