youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228)

Browse Source 
Signed-off-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
This commit is contained in:
Sanket Kale
2024-11-26 08:02:39 +05:30
committed by GitHub
parent 45ac4ff270
commit a6760f6456
9 changed files with 678 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/offline_inference.py
View File

@ -19,4 +19,4 @@ outputs = llm.generate(prompts, sampling_params)
for output in outputs:
prompt = output.prompt
generated_text = output.outputs[0].text
print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")