Skip to content

Conversation

@roomote
Copy link
Contributor

@roomote roomote bot commented Nov 25, 2025

This PR attempts to address Issue #9551 regarding Kimi K2 Thinking native tool calls not working properly with ik_llama.cpp and other OpenAI-compatible servers.

Problem

When using Kimi K2 with ik_llama.cpp, tool calls were being detected but the results were not being sent back to the model properly. The issue was that:

  1. The server might not set finish_reason to "tool_calls" specifically
  2. Tool calls were accumulated but not always yielded correctly

Solution

Modified base-openai-compatible-provider.ts to:

  • Yield tool calls when ANY finish_reason is present (not just "tool_calls")
  • Added fallback to yield tool calls at stream end if not already yielded
  • Added tracking with hasYieldedToolCalls flag to prevent duplicates
  • Ensures tool calls with valid ID and name are always yielded

Testing

  • Existing tests pass
  • The fix should work with various OpenAI-compatible servers including ik_llama.cpp

Feedback and guidance are welcome!


Important

Fixes tool call handling in base-openai-compatible-provider.ts for OpenAI-compatible servers by yielding tool calls on any finish_reason and adding a fallback mechanism.

  • Behavior:
    • In base-openai-compatible-provider.ts, tool calls are now yielded when any finish_reason is present, not just "tool_calls".
    • Adds a fallback to yield tool calls at stream end if not already yielded.
    • Introduces hasEmittedToolCallEnd flag to prevent duplicate tool call end events.
  • Testing:
    • Existing tests pass, ensuring compatibility with various OpenAI-compatible servers including ik_llama.cpp.

This description was created by Ellipsis for 2177a8c. You can customize this summary. It will automatically update as commits are pushed.

@roomote roomote bot requested review from cte, jr and mrubens as code owners November 25, 2025 03:26
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working labels Nov 25, 2025
@roomote
Copy link
Contributor Author

roomote bot commented Nov 25, 2025

Rooviewer Clock   See task on Roo Cloud

Review complete. No issues found. The previous concern about inconsistent argument handling is no longer applicable - the code now uses tool_call_partial + tool_call_end events instead of accumulated tool_call events, and the type definitions allow optional arguments for partial chunks.

  • Fix inconsistent handling of empty arguments in main tool call yielding path (No longer applicable - code restructured)
Previous reviews

Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues.

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Nov 25, 2025
@MobCode100
Copy link

MobCode100 commented Dec 5, 2025

Tested this PR, no issues with Roo Code v3.35.5. Thanks for the PR.

@hannesrudolph
Copy link
Collaborator

@roomote can you please run merge resolver mode on this?

…_llama.cpp

- Modified base-openai-compatible-provider.ts to better handle tool calls
- Tool calls are now yielded when any finish_reason is present, not just "tool_calls"
- Added fallback to yield tool calls at stream end if not already yielded
- Added tracking to prevent duplicate tool call yields
- Ensures tool calls with valid ID and name are always yielded
- Fixes issue #9551 where Kimi K2 Thinking native tool calls were not working
@roomote roomote bot force-pushed the fix-kimi-k2-tool-calls branch from f310921 to 2177a8c Compare January 7, 2026 01:23
@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Jan 7, 2026
@roomote
Copy link
Contributor Author

roomote bot commented Jan 7, 2026

Fixaroo Clock   See task on Roo Cloud

Successfully resolved merge conflicts for PR #9557 "Fix Kimi K2 Thinking native tool calls for OpenAI-compatible servers".

Resolution Summary:

  • Adapted the PR's intent to main's new streaming approach that uses tool_call_partial and tool_call_end events
  • Modified tool call end emission to trigger on ANY finish_reason (not just "tool_calls") for compatibility with servers like ik_llama.cpp
  • Added fallback logic at stream end to emit tool_call_end if not already emitted
  • Added hasEmittedToolCallEnd flag to prevent duplicate emissions

All type checks passed. The rebase completed successfully and the branch is now up-to-date with main.

View commit | Revert commit

@hannesrudolph hannesrudolph moved this from Triage to PR [Needs Prelim Review] in Roo Code Roadmap Jan 7, 2026
@hannesrudolph hannesrudolph added PR - Needs Preliminary Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Jan 7, 2026
@daniel-lxs
Copy link
Member

This issue was already fixed by PR #9542 which was merged the same day this PR was opened.

The real fix added NativeToolCallParser.finalizeRawChunks() which is called at the end of stream processing regardless of finish_reason. This handles the case where servers don't send finish_reason = "tool_calls" more elegantly at the consumer level.

This PR modifies the provider layer to emit tool_call_end events, but those events aren't actually consumed by Task.ts - there's no case handler for them in the stream processing switch. The events emitted by this code would be silently ignored.

Closing since the underlying issue was resolved by #9542.

@daniel-lxs daniel-lxs closed this Jan 7, 2026
@github-project-automation github-project-automation bot moved this from PR [Needs Prelim Review] to Done in Roo Code Roadmap Jan 7, 2026
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Jan 7, 2026
@daniel-lxs daniel-lxs deleted the fix-kimi-k2-tool-calls branch January 7, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working PR - Needs Preliminary Review size:S This PR changes 10-29 lines, ignoring generated files.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants