youngkingdom/vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Bugfix] Fix tool call template validation for Mistral models (#17644)

Browse Source 
Signed-off-by: Rick Yuan <yuan821120@gmail.com>
Signed-off-by: RIck Yuan <yuan821120@gmail.com>
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
This commit is contained in:
Rick Yuan
2025-05-08 17:47:19 +08:00
committed by GitHub
parent 0a9bbaa104
commit ca04b97c93
1 changed files with 9 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
examples/tool_chat_template_mistral3.jinja
View File

@ -29,7 +29,14 @@
{%- set user_messages = loop_messages | selectattr("role", "equalto", "user") | list %}
{%- for message in loop_messages | rejectattr("role", "equalto", "tool") | rejectattr("role", "equalto", "tool_results") | selectattr("tool_calls", "undefined") %}
{%- set filtered_messages = [] %}
{%- for message in loop_messages %}
{%- if message["role"] not in ["tool", "tool_results"] and not message.get("tool_calls") %}
{%- set filtered_messages = filtered_messages + [message] %}
{%- endif %}
{%- endfor %}
{%- for message in filtered_messages %}
{%- if (message["role"] == "user") != (loop.index0 % 2 == 0) %}
{{- raise_exception("After the optional system message, conversation roles must alternate user/assistant/user/assistant/...") }}
{%- endif %}
@ -116,4 +123,4 @@
{%- else %}
{{- raise_exception("Only user and assistant roles are supported, with the exception of an initial optional system message!") }}
{%- endif %}
{%- endfor %}
{%- endfor %}