Skip to content

[serve] Forward tool_calls/tool_call_id in processor inputs#45418

Merged
SunMarc merged 4 commits intomainfrom
forward-tool-use
Apr 15, 2026
Merged

[serve] Forward tool_calls/tool_call_id in processor inputs#45418
SunMarc merged 4 commits intomainfrom
forward-tool-use

Conversation

@qgallouedec
Copy link
Copy Markdown
Member

  • get_processor_inputs_from_messages builds a new dict per message with only role and content, dropping tool_calls and tool_call_id. This means apply_chat_template can't properly render multi-turn tool-use conversations.
  • Forward these fields so the chat template sees the full tool-use context.

Example

Multi-turn tool-use conversation sent to transformers serve:

from transformers.cli.serving.utils import BaseHandler, Modality

messages = [
    {"role": "user", "content": "What's the weather in Paris?"},
    {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": '{"city": "Paris"}'}}]},
    {"role": "tool", "content": "22°C, sunny", "tool_call_id": "call_1"},
]
print(BaseHandler.get_processor_inputs_from_messages(messages, Modality.LLM))

Before this PR, tool_calls and tool_call_id were dropped:

[
    {'role': 'user', 'content': "What's the weather in Paris?"},
    {'role': 'assistant', 'content': []},
    {'role': 'tool', 'content': '22°C, sunny'}
]

After this PR, they are forwarded to apply_chat_template:

[
    {'role': 'user', 'content': "What's the weather in Paris?"},
    {'role': 'assistant', 'content': [], 'tool_calls': [{'id': 'call_1', 'type': 'function', 'function': {'name': 'get_weather', 'arguments': '{"city": "Paris"}'}}]},
    {'role': 'tool', 'content': '22°C, sunny', 'tool_call_id': 'call_1'}
]

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Comment thread src/transformers/cli/serving/utils.py
Copy link
Copy Markdown
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ! cc @Rocketknight1 for confirmation

@SunMarc
Copy link
Copy Markdown
Member

SunMarc commented Apr 15, 2026

@bot /style

@SunMarc SunMarc enabled auto-merge April 15, 2026 20:22
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 15, 2026

Style fix bot fixed some files and pushed the changes.

Comment thread src/transformers/cli/serving/utils.py Outdated
@SunMarc SunMarc added this pull request to the merge queue Apr 15, 2026
Merged via the queue into main with commit 8426e7e Apr 15, 2026
18 checks passed
@SunMarc SunMarc deleted the forward-tool-use branch April 15, 2026 20:57
shaun0927 added a commit to shaun0927/transformers that referenced this pull request Apr 17, 2026
OpenAI-compatible tool-calling payloads may encode assistant tool-call
messages with explicit null content rather than omitting the field.
The serve path already handled missing content, but iterated over None
for the explicit-null case and raised TypeError before the tool calls
reached apply_chat_template.

This keeps the fix intentionally small: normalize None to an empty
content list and pin the behavior with focused LLM/VLM regression tests.

Constraint: Recent serve/tool-call fixes (huggingface#45348, huggingface#45418, huggingface#45463) make compatibility regressions in this path especially review-sensitive
Rejected: Broader serve content refactor | unnecessary scope for a one-line crash fix
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep follow-up discussion focused on the explicit null-content case; do not mix unrelated serve cleanup into the PR
Tested: Direct local repro before/after; pytest -q tests/cli/test_serve.py -k 'tool_use_fields_forwarded'
Not-tested: Full repo CI; end-to-end HTTP serve session
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants