[Model] Add base class for LoRA-supported models (#5018)
This commit is contained in:
@ -4,6 +4,9 @@ Using LoRA adapters
|
||||
===================
|
||||
|
||||
This document shows you how to use `LoRA adapters <https://arxiv.org/abs/2106.09685>`_ with vLLM on top of a base model.
|
||||
|
||||
LoRA adapters can be used with any vLLM model that implements :class:`~vllm.model_executor.models.interfaces.SupportsLoRA`.
|
||||
|
||||
Adapters can be efficiently served on a per request basis with minimal overhead. First we download the adapter(s) and save
|
||||
them locally with
|
||||
|
||||
|
||||
Reference in New Issue
Block a user