feat: add function calling / tool use support and display#46
Conversation
- Add type definitions for tool calls, tool definitions, and message formats - Record tools, tool_choice, and tool_calls in completions log - Extract tool calls from both streaming and non-streaming responses - Add frontend components to display tool calls and tool definitions - Support tool message display with tool_call_id reference - Add i18n translations for tool-related UI elements (en-US, zh-CN) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 Walkthrough遍历此次变更在后端与前端引入并扩展了 OpenAI 风格的工具调用支持:后端新增并导出与工具调用相关的类型、在完成端点的流式与非流式路径中检测、收集并附加 变更
序列图sequenceDiagram
participant Client as Client
participant API as "完成 API 端点"
participant Model as "模型/提供者"
participant Logger as "日志/数据库"
Client->>API: 发送请求(包含 messages、tools、tool_choice)
API->>Logger: buildCompletionRecord(初始记录含工具元数据)
API->>Model: 调用模型(可能返回 tool_use 块 或 流式增量)
alt 非流式
Model->>API: 返回完成(可能含 tool_use 或 tool_calls)
API->>API: extractToolCalls -> ToolCallType[]
API->>Client: 返回完成(包含 content 与 tool_calls)
else 流式
Model->>API: 流式发送增量(含 token 与 tool_use 块)
API->>API: 跟踪 streamToolCalls 与 toolCallArguments(收集 JSON 参数)
API->>API: 最终化并附加 tool_calls 到完成记录
API->>Client: 发送完整流式响应(最终事件包含 tool_calls)
end
API->>Logger: 记录最终完成(含工具信息与 tool_calls)
sequenceDiagram
participant Browser as 浏览器
participant DetailPanel as 详情面板
participant PrettyView as PrettyView 组件
participant I18n as 国际化模块
Browser->>DetailPanel: 打开请求详情(含 tool_calls/tools/tool_choice)
DetailPanel->>PrettyView: 传入请求数据
PrettyView->>I18n: 读取翻译键(ToolCalls/Tools/ToolChoice 等)
I18n-->>PrettyView: 返回本地化字符串
PrettyView->>PrettyView: 渲染消息与响应(显示 tool_calls、NoContent 占位)
PrettyView->>PrettyView: 渲染 ToolCallDisplay(解析并展示 arguments/id/name)
PrettyView->>PrettyView: 渲染 ToolsDefinitionDisplay(描述与参数)
PrettyView->>Browser: 在详情面板中呈现工具相关信息
预估代码审查工作量🎯 3 (Moderate) | ⏱️ ~25 minutes 诗歌
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
Summary of ChangesHello @pescn, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly upgrades the system's ability to handle and visualize OpenAI-compatible function calling and tool use. It introduces new data structures and logic to accurately log all aspects of tool interactions, from initial definitions and choice settings in requests to the actual tool calls made in responses. The corresponding frontend updates provide a clear, interactive display of these details, making it easier to understand how AI models are leveraging external tools. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request adds comprehensive support for OpenAI-compatible function calling and tool use. The changes span both the backend and frontend, including schema updates, API handling for streaming and non-streaming tool calls, and new UI components for displaying tool information.
The implementation is well-done. I've found a bug in the frontend related to parsing tool arguments which could lead to a type error. I've also suggested some refactoring in the backend to improve type safety and reduce code duplication by extracting the tool_choice type definition.
Overall, great work on adding this significant feature.
| parsedArgs = typeof toolCall.function.arguments === 'string' | ||
| ? JSON.parse(toolCall.function.arguments) | ||
| : toolCall.function.arguments |
There was a problem hiding this comment.
There's a type mismatch here. The ToolCall interface defines function.arguments as a string. However, the else branch of this ternary operator attempts to assign this string value directly to parsedArgs, which is typed as Record<string, unknown> | null. This will cause a TypeScript compilation error.
Since toolCall.function.arguments is always a string according to its type, the typeof check and the else branch are unnecessary. You can simplify this to just parse the string.
| parsedArgs = typeof toolCall.function.arguments === 'string' | |
| ? JSON.parse(toolCall.function.arguments) | |
| : toolCall.function.arguments | |
| parsedArgs = JSON.parse(toolCall.function.arguments) |
| messages: Array<{ role: string; content: string }>, | ||
| messages: CompletionsMessageType[], | ||
| tools?: ToolDefinitionType[], | ||
| toolChoice?: "auto" | "none" | "required" | { type: "function"; function: { name: string } }, |
There was a problem hiding this comment.
This complex type for toolChoice is duplicated in a few places. To improve reusability and maintainability, I recommend extracting it into its own exported ToolChoiceType in backend/src/db/schema.ts and then using it here. This will make the code cleaner and easier to maintain.
You'll also need to import ToolChoiceType from @/db/schema.
| toolChoice?: "auto" | "none" | "required" | { type: "function"; function: { name: string } }, | |
| toolChoice?: ToolChoiceType, |
| const toolChoice = rawBody.tool_choice as | ||
| | "auto" | ||
| | "none" | ||
| | "required" | ||
| | { type: "function"; function: { name: string } } | ||
| | undefined; |
There was a problem hiding this comment.
Similar to the buildCompletionRecord function, this type assertion can be simplified by using a reusable ToolChoiceType (which I've suggested extracting in backend/src/db/schema.ts). This improves readability and avoids type duplication.
const toolChoice = rawBody.tool_choice as ToolChoiceType | undefined;|
Note Docstrings generation - SUCCESS |
Docstrings generation was requested by @pescn. * #46 (comment) The following files were modified: * `backend/src/api/v1/completions.ts` * `backend/src/utils/api-helpers.ts` * `frontend/src/pages/requests/detail-panel/pretty-view.tsx`
- Extract ToolChoiceType to schema.ts for reusability - Use ToolChoiceType in completions.ts to reduce type duplication - Simplify JSON.parse in ToolCallDisplay (arguments is always string) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive support for OpenAI-compatible function calling and tool use, with changes across the backend and frontend. The backend is updated to log tool definitions, choices, and calls, including robust handling for both streaming and non-streaming responses. The frontend adds new UI components to display this information clearly in the request detail panel. My review identifies a critical issue in the backend's streaming logic that could lead to data corruption and a medium-severity issue on the frontend related to type safety and code clarity. Overall, this is a significant and well-implemented feature, and addressing these points will enhance its robustness.
| if (chunk.type === "content_block_start") { | ||
| // Track new tool call block | ||
| if (chunk.contentBlock?.type === "tool_use") { | ||
| const index = chunk.index ?? 0; | ||
| streamToolCalls.set(index, { | ||
| id: chunk.contentBlock.id, | ||
| type: "function", | ||
| function: { | ||
| name: chunk.contentBlock.name, | ||
| arguments: "", | ||
| }, | ||
| }); | ||
| toolCallArguments.set(index, []); | ||
| } | ||
| } else if (chunk.type === "content_block_delta") { | ||
| if (chunk.delta?.type === "text_delta" && chunk.delta.text) { | ||
| textParts.push(chunk.delta.text); | ||
| } else if ( | ||
| chunk.delta?.type === "thinking_delta" && | ||
| chunk.delta.thinking | ||
| ) { | ||
| thinkingParts.push(chunk.delta.thinking); | ||
| } else if (chunk.delta?.type === "input_json_delta" && chunk.delta.partialJson) { | ||
| // Collect tool call arguments | ||
| const index = chunk.index ?? 0; | ||
| const args = toolCallArguments.get(index); | ||
| if (args) { | ||
| args.push(chunk.delta.partialJson); | ||
| } | ||
| } | ||
| } else if (chunk.type === "content_block_stop") { | ||
| // Finalize tool call arguments | ||
| const index = chunk.index ?? 0; | ||
| const toolCall = streamToolCalls.get(index); | ||
| const args = toolCallArguments.get(index); | ||
| if (toolCall && args) { | ||
| toolCall.function.arguments = args.join(""); | ||
| } | ||
| } |
There was a problem hiding this comment.
The use of chunk.index ?? 0 to get the tool call index can lead to data corruption when processing streamed responses with multiple tool calls. If an adapter fails to provide an index for a tool call chunk (i.e., chunk.index is undefined), it will default to 0. This could cause data for a later tool call to overwrite or mix with data for the first tool call at index 0, resulting in incorrectly logged completion data.
It would be safer to validate that chunk.index is a number when dealing with tool call chunks and log an error if it's missing.
// Collect content for completion record
if (chunk.type === "content_block_start") {
// Track new tool call block
if (chunk.contentBlock?.type === "tool_use") {
if (typeof chunk.index !== "number") {
logger.error("Missing index for tool_use content_block_start in stream");
continue;
}
const index = chunk.index;
streamToolCalls.set(index, {
id: chunk.contentBlock.id,
type: "function",
function: {
name: chunk.contentBlock.name,
arguments: "",
},
});
toolCallArguments.set(index, []);
}
} else if (chunk.type === "content_block_delta") {
if (chunk.delta?.type === "text_delta" && chunk.delta.text) {
textParts.push(chunk.delta.text);
} else if (
chunk.delta?.type === "thinking_delta" &&
chunk.delta.thinking
) {
thinkingParts.push(chunk.delta.thinking);
} else if (chunk.delta?.type === "input_json_delta" && chunk.delta.partialJson) {
// Collect tool call arguments
if (typeof chunk.index !== "number") {
logger.error("Missing index for input_json_delta in stream");
continue;
}
const index = chunk.index;
const args = toolCallArguments.get(index);
if (args) {
args.push(chunk.delta.partialJson);
}
}
} else if (chunk.type === "content_block_stop") {
// Finalize tool call arguments
if (typeof chunk.index !== "number") {
logger.error("Missing index for content_block_stop in stream");
continue;
}
const index = chunk.index;
const toolCall = streamToolCalls.get(index);
const args = toolCallArguments.get(index);
if (toolCall && args) {
toolCall.function.arguments = args.join("");
}
}| // Check if this is a tool result message | ||
| const isToolMessage = message.role === 'tool' | ||
| const toolCallId = 'tool_call_id' in message ? (message as { tool_call_id?: string }).tool_call_id : undefined | ||
|
|
||
| // Check if this is an assistant message with tool calls | ||
| const toolCalls = 'tool_calls' in message ? (message as { tool_calls?: ToolCall[] }).tool_calls : undefined |
There was a problem hiding this comment.
The property access for tool_call_id and tool_calls is a bit verbose with repeated in checks and type casting. This can be made more concise and readable by casting the message object once to an extended type. This improves maintainability by centralizing the type assertion.
| // Check if this is a tool result message | |
| const isToolMessage = message.role === 'tool' | |
| const toolCallId = 'tool_call_id' in message ? (message as { tool_call_id?: string }).tool_call_id : undefined | |
| // Check if this is an assistant message with tool calls | |
| const toolCalls = 'tool_calls' in message ? (message as { tool_calls?: ToolCall[] }).tool_calls : undefined | |
| const extendedMessage = message as RequestMessage & { tool_call_id?: string; tool_calls?: ToolCall[] }; | |
| const isToolMessage = extendedMessage.role === 'tool' | |
| const toolCallId = extendedMessage.tool_call_id | |
| const toolCalls = extendedMessage.tool_calls |
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
frontend/src/pages/requests/detail-panel/pretty-view.tsx (1)
279-366: 为兼容旧记录,添加从 extraBody 的回退读取第 279 行的注释说明旧记录可能在
extraBody中存储 tools/tool_choice,但当前实现仅从promptData.tools和promptData.tool_choice读取。建议添加回退读取以确保兼容性:- const tools = promptData.tools - const toolChoice = promptData.tool_choice + const tools = + promptData.tools ?? (promptData.extraBody?.tools as ToolDefinition[] | undefined) + const toolChoice = + promptData.tool_choice ?? (promptData.extraBody?.tool_choice as unknown)
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
backend/src/api/v1/completions.tsbackend/src/db/schema.tsfrontend/src/pages/requests/detail-panel/pretty-view.tsx
🧰 Additional context used
🧬 Code graph analysis (2)
backend/src/api/v1/completions.ts (2)
backend/src/db/schema.ts (4)
CompletionsMessageType(105-114)ToolDefinitionType(77-84)ToolChoiceType(89-93)ToolCallType(65-72)backend/src/utils/api-helpers.ts (2)
extractToolCalls(145-164)extractContentText(121-139)
frontend/src/pages/requests/detail-panel/pretty-view.tsx (4)
frontend/src/lib/content.ts (1)
extractReasoning(1-13)frontend/src/components/app/markdown/index.tsx (1)
Markdown(11-28)frontend/src/components/ui/indicator-badge.tsx (1)
IndicatorBadge(6-16)frontend/src/components/ui/collapsible.tsx (3)
Collapsible(29-29)CollapsibleTrigger(29-29)CollapsibleContent(29-29)
🔇 Additional comments (13)
frontend/src/pages/requests/detail-panel/pretty-view.tsx (6)
2-11: 图标导入变更合理
新增的图标与工具相关 UI 保持一致,没看到问题。
30-48: 工具相关类型定义清晰
类型结构与 OpenAI 格式一致,便于前端展示与解析。
121-158: 工具消息与工具调用展示处理到位
工具消息标识与工具调用列表的渲染逻辑清晰,交互一致。
162-216: Response 渲染逻辑完整
工具调用区块与无内容占位处理都符合预期。
462-500: 工具调用参数展示健壮
JSON 解析失败回退到原始字符串的处理合理。
503-548: 工具定义展示实现 OK
折叠展示参数与描述的结构清晰、易读。backend/src/db/schema.ts (1)
62-134: 类型扩展与工具调用格式对齐
新增类型与 prompt/completion 结构扩展合理,便于后续复用与序列化。backend/src/api/v1/completions.ts (6)
18-29: 类型与工具提取导入一致
新导入的类型与工具调用解析函数与新增逻辑匹配良好。
65-85: 记录结构扩展到 tools/tool_choice 是正确方向
prompt 里补全工具元数据后,日志可追踪性更好。
173-181: 非流式响应补充 tool_calls 正确
从内部响应提取工具调用并写入 completion 的处理合理。
297-349: 流式工具调用追踪实现合理
按 index 聚合 tool_use 与参数片段的逻辑清晰。
369-387: 流式最终 tool_calls 汇总 OK
最终 completion 里补齐 tool_calls 与内容组合逻辑一致。
534-548: 请求体工具元数据抽取用于记录 OK
直接从 rawBody 取 tools/tool_choice 并写入日志,符合记录目的。
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
- Fix streaming tool call tracking to use tool ID as key instead of index to prevent data corruption when multiple tool calls are processed - Simplify type access in MessageContent by casting message once - Add fallback to extraBody for backward compatibility with older records that may have tools/tool_choice stored in extraBody Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive support for function calling/tool use, a significant feature enhancement. The changes span both the backend and frontend, enabling the logging and display of tool definitions, choices, and calls. The backend modifications correctly update the database schema and API logic for both streaming and non-streaming responses. The frontend adds well-designed collapsible UI components to display this new information.
My review identifies a potential robustness issue in the stream processing logic where a risky fallback could lead to data corruption in certain edge cases. I've provided suggestions to make the implementation more resilient. Overall, this is a solid contribution that greatly expands the gateway's capabilities.
| const index = chunk.index ?? 0; | ||
| const toolId = indexToIdMap.get(index); | ||
| if (toolId) { | ||
| const args = toolCallArguments.get(toolId); | ||
| if (args) { | ||
| args.push(chunk.delta.partialJson); | ||
| } | ||
| } |
There was a problem hiding this comment.
The fallback ?? 0 for chunk.index is risky. If a provider sends a delta for a tool call other than the first one and omits the index, this logic will incorrectly associate the delta with the tool call at index 0. This could lead to corrupted tool call arguments.
While the current adapters might always provide an index, this makes the stream processing logic fragile. It's safer to handle the undefined case explicitly by logging a warning and skipping the chunk. I've also slightly simplified the nested checks.
const index = chunk.index;
if (index === undefined) {
logger.warn("Tool call delta chunk is missing index, skipping.", { chunk });
continue;
}
const toolId = indexToIdMap.get(index);
const args = toolId ? toolCallArguments.get(toolId) : undefined;
if (args) {
args.push(chunk.delta.partialJson);
}| const index = chunk.index ?? 0; | ||
| const toolId = indexToIdMap.get(index); | ||
| if (toolId) { | ||
| const toolCall = streamToolCalls.get(toolId); | ||
| const args = toolCallArguments.get(toolId); | ||
| if (toolCall && args) { | ||
| toolCall.function.arguments = args.join(""); | ||
| } | ||
| } |
There was a problem hiding this comment.
Similar to the content_block_delta case, the fallback ?? 0 for chunk.index is risky here. If an index is missing on a stop chunk for a multi-tool-call response, it could lead to arguments of one tool call not being finalized correctly.
To make the logic more robust, we should explicitly check for an undefined index and handle it gracefully.
const index = chunk.index;
if (index === undefined) {
logger.warn("Tool call stop chunk is missing index, skipping.", { chunk });
continue;
}
const toolId = indexToIdMap.get(index);
const toolCall = toolId ? streamToolCalls.get(toolId) : undefined;
const args = toolId ? toolCallArguments.get(toolId) : undefined;
if (toolCall && args) {
toolCall.function.arguments = args.join("");
}Instead of defaulting to index 0 which could cause data corruption, now explicitly check if chunk.index is defined before processing input_json_delta and content_block_stop chunks. Log a warning when index is missing for input_json_delta. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This is an excellent pull request that adds comprehensive support for function calling and tool use, a significant feature. The changes are well-structured across the backend and frontend, handling both streaming and non-streaming responses, and even considering backward compatibility. The new UI components for displaying tool calls and definitions are a great addition. My main feedback is to improve the request validation schema on the backend to ensure type safety and robustness for the new fields.
| const rawBody = body as Record<string, unknown>; | ||
| const tools = rawBody.tools as ToolDefinitionType[] | undefined; | ||
| const toolChoice = rawBody.tool_choice as ToolChoiceType | undefined; | ||
|
|
||
| // Build completion record for logging (with full message data) | ||
| const completion = buildCompletionRecord( | ||
| body.model, | ||
| modelConfig.id, | ||
| body.messages, | ||
| rawBody.messages as CompletionsMessageType[], |
There was a problem hiding this comment.
These type assertions on rawBody are unsafe as they bypass the request validation logic. This is happening because the tChatCompletionCreate schema (defined on line 46) has not been updated to include the new tools, tool_choice fields, or the more complex structure for messages.
To ensure type safety and prevent potential runtime errors from malformed requests, it's crucial to update the tChatCompletionCreate Elysia schema to accurately reflect the expected request body. This would make these type assertions unnecessary and leverage the framework's validation capabilities.
Update tChatCompletionCreate schema to include: - tToolDefinition: validates tool function definitions - tToolChoice: validates tool choice (auto/none/required/function) - tMessage: supports tool_calls, tool_call_id, and name fields This removes unsafe type assertions on rawBody and leverages the framework's validation capabilities for type safety. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request adds comprehensive support for OpenAI-compatible function calling and tool use. The backend changes correctly handle tool definitions, choices, and calls for both streaming and non-streaming responses, with well-defined database schema and API validation updates. The frontend is also updated to display this new information in a user-friendly, collapsible UI. My review includes a suggestion to improve the robustness of tool argument serialization on the backend and a fix for the frontend to avoid displaying tool information redundantly. Overall, this is a solid implementation of a complex feature.
| toolCalls.push({ | ||
| id: block.id, | ||
| type: "function", | ||
| function: { | ||
| name: block.name, | ||
| arguments: JSON.stringify(block.input), | ||
| }, | ||
| }); |
There was a problem hiding this comment.
JSON.stringify can throw an exception if block.input contains circular references. While this is unlikely for tool arguments from an LLM, wrapping this operation in a try-catch block would make the function more robust. This would prevent the entire request from failing due to a single malformed tool call, instead just logging the error and skipping that tool call.
try {
const args = JSON.stringify(block.input);
toolCalls.push({
id: block.id,
type: "function",
function: {
name: block.name,
arguments: args,
},
});
} catch (error) {
logger.error("Failed to stringify tool call arguments for tool_use block", {
toolName: block.name,
toolId: block.id,
error,
});
}| key: 'extraBody', | ||
| name: t('pages.requests.detail-panel.pretty-view.ExtraBody'), | ||
| value: <ExtraDataDisplay data={data.prompt.extraBody} />, | ||
| hidden: !data.prompt.extraBody, | ||
| value: <ExtraDataDisplay data={promptData.extraBody} />, | ||
| hidden: !promptData.extraBody, | ||
| fullWidth: true, | ||
| }, |
There was a problem hiding this comment.
To avoid displaying tools and tool_choice in both their dedicated sections and the 'Extra body' section, it's better to filter them out from the extraBody before rendering. This is particularly relevant for older records where these properties might exist inside extraBody. The current implementation could lead to redundant information in the UI.
key: 'extraBody',
name: t('pages.requests.detail-panel.pretty-view.ExtraBody'),
value: (
<ExtraDataDisplay
data={
promptData.extraBody
? (({ tools: _t, tool_choice: _tc, ...rest }) => rest)(promptData.extraBody)
: undefined
}
/>
),
hidden:
!promptData.extraBody ||
Object.keys(promptData.extraBody).filter((k) => k !== 'tools' && k !== 'tool_choice').length === 0,
fullWidth: true,
},
Review ResponseRegarding
|
Summary
tools), tool choice settings (tool_choice), and tool call responses (tool_calls) in the completions databaseChanges
Backend
ToolCallType,ToolDefinitionType,MessageContentPartType, andCompletionsMessageTypeextractToolCalls()utility functionFrontend
ToolCallDisplayandToolsDefinitionDisplaycomponents with collapsible panelsTest plan
🤖 Generated with Claude Code
Summary by CodeRabbit
发布说明
✏️ Tip: You can customize this high-level summary in your review settings.