[Doc]Add documentation to benchmarking script when running TGI (#4920)

This commit is contained in:
Kuntai Du
2024-05-20 13:16:57 -07:00
committed by GitHub
parent 1937e29848
commit c3af44722c
2 changed files with 5 additions and 1 deletions

View File

@ -17,6 +17,10 @@ On the client side, run:
--dataset-path <path to dataset> \
--request-rate <request_rate> \ # By default <request_rate> is inf
--num-prompts <num_prompts> # By default <num_prompts> is 1000
when using tgi backend, add
--endpoint /generate_stream
to the end of the command above.
"""
import argparse
import asyncio