Skip to content

fix(provider): handle delayed tool call names in streamed responses#18623

Open
Haohao-end wants to merge 1 commit intoanomalyco:devfrom
Haohao-end:fix/16488-vllm-xml-tool-stream
Open

fix(provider): handle delayed tool call names in streamed responses#18623
Haohao-end wants to merge 1 commit intoanomalyco:devfrom
Haohao-end:fix/16488-vllm-xml-tool-stream

Conversation

@Haohao-end
Copy link
Copy Markdown

Issue for this PR

Fixes #16488

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

This fixes a streamed tool-call parsing issue in the OpenAI-compatible chat model path.

Some OpenAI-compatible providers can stream tool_calls where the first delta includes the call id and partial arguments, but function.name arrives in a later chunk. The current parser treated a missing function.name in the first chunk as a fatal error and aborted the loop.

This change makes the parser buffer incomplete tool call deltas until the tool name is available. Once the name arrives, it emits the normal tool input events and continues with the existing completion flow. I also kept the flush behavior aligned with the previous implementation so normal providers should not change behavior.

I added:

  • a regression test for the delayed function.name streaming case
  • a session-level test covering interrupted pending/running tool calls being converted into valid error results

How did you verify your code works?

I tested the changes locally with:

  • bun test test/provider/copilot/copilot-chat-model.test.ts
  • bun test test/session/message-v2.test.ts
  • bun typecheck

I also pushed with the repo's pre-push checks enabled, and bun turbo typecheck completed successfully.

Screenshots / recordings

N/A — no UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Silent agent crash and subsequent 400 Bad Request with local vLLM (Qwen 3 XML Tools)

1 participant