Skip to content

feat: add function calling / tool use support and display#46

Merged
pescn merged 5 commits into
mainfrom
feat/function-calling-support
Jan 19, 2026
Merged

feat: add function calling / tool use support and display#46
pescn merged 5 commits into
mainfrom
feat/function-calling-support

Conversation

@pescn
Copy link
Copy Markdown
Contributor

@pescn pescn commented Jan 19, 2026

Summary

  • Add comprehensive support for OpenAI-compatible function calling / tool use in request logging
  • Record complete tool definitions (tools), tool choice settings (tool_choice), and tool call responses (tool_calls) in the completions database
  • Display tool calls and tool definitions in the request detail panel with collapsible UI components
  • Support both streaming and non-streaming response formats for tool call extraction

Changes

Backend

  • schema.ts: Add type definitions for ToolCallType, ToolDefinitionType, MessageContentPartType, and CompletionsMessageType
  • completions.ts: Extract and record tools, tool_choice, and tool_calls from requests/responses
  • api-helpers.ts: Add extractToolCalls() utility function

Frontend

  • pretty-view.tsx: Add ToolCallDisplay and ToolsDefinitionDisplay components with collapsible panels
  • i18n: Add translations for tool-related UI elements (en-US, zh-CN)

Test plan

  • Send a chat completion request with tools defined and verify they are recorded in the database
  • Verify tool calls in response are captured for both streaming and non-streaming modes
  • Check the request detail panel displays tool definitions and tool calls correctly
  • Verify the UI works in both English and Chinese locales

🤖 Generated with Claude Code

Summary by CodeRabbit

发布说明

  • 新功能
    • 后端记录并返回工具调用信息、工具列表与工具选择,流式与非流式响应均附带工具调用。
  • 改进
    • 响应内容可为无文本但含工具调用时显示“无内容”占位。
    • 请求详情“漂亮视图”新增可展开的工具调用与工具定义面板,并显示调用 ID、名称与参数(尝试解析 JSON,失败回退原始文本)。
    • 补充中/英两种语言的界面文案。

✏️ Tip: You can customize this high-level summary in your review settings.

- Add type definitions for tool calls, tool definitions, and message formats
- Record tools, tool_choice, and tool_calls in completions log
- Extract tool calls from both streaming and non-streaming responses
- Add frontend components to display tool calls and tool definitions
- Support tool message display with tool_call_id reference
- Add i18n translations for tool-related UI elements (en-US, zh-CN)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jan 19, 2026

Warning

Rate limit exceeded

@pescn has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 4 minutes and 8 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 24baa63 and 7adbd6d.

📒 Files selected for processing (1)
  • backend/src/api/v1/completions.ts

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

📝 Walkthrough

遍历

此次变更在后端与前端引入并扩展了 OpenAI 风格的工具调用支持:后端新增并导出与工具调用相关的类型、在完成端点的流式与非流式路径中检测、收集并附加 tool_calls;前端新增国际化键并在请求详情面板可视化工具定义与工具调用详情(含参数与占位显示)。

变更

群组 / 文件(s) 变更摘要
后端类型与模式
backend/src/db/schema.ts
新增并导出类型:ToolCallTypeToolDefinitionTypeToolChoiceTypeMessageContentPartTypeCompletionsMessageType;将 CompletionsPromptType.messages 改为 CompletionsMessageType[],新增 toolstool_choice 字段;允许 CompletionsCompletionType.contentnull 并可包含 tool_calls[]
后端完成端点逻辑
backend/src/api/v1/completions.ts
扩展 buildCompletionRecord 签名以接收 messages: CompletionsMessageType[]、可选 tools?: ToolDefinitionType[]toolChoice?: ToolChoiceType;在非流与流式路径中集成 extractToolCalls、引入 streamToolCallstoolCallArguments 跟踪流式工具使用,检测 tool_use 块并在最终完成中附加 tool_calls;使用原始请求体记录 toolstool_choice
后端工具提取辅助
backend/src/utils/api-helpers.ts
新增导出函数 `extractToolCalls(response: InternalResponse): ToolCallType[]
前端国际化
frontend/src/i18n/locales/en-US.json,
frontend/src/i18n/locales/zh-CN.json
新增请求详情面板使用的工具相关翻译键(例如:ToolCalls、Tools、ToolChoice、Arguments、Description、Parameters、NoContent)。
前端请求详情 UI
frontend/src/pages/requests/detail-panel/pretty-view.tsx
添加 ToolCallDisplayToolsDefinitionDisplay 组件;在消息与响应渲染中支持显示工具调用(可折叠参数视图、tool_call_id 徽章、工具定义面板);处理无内容/无工具调用的占位显示;对工具参数尝试 JSON 解析并在失败时回退到原始字符串。

序列图

sequenceDiagram
    participant Client as Client
    participant API as "完成 API 端点"
    participant Model as "模型/提供者"
    participant Logger as "日志/数据库"

    Client->>API: 发送请求(包含 messages、tools、tool_choice)
    API->>Logger: buildCompletionRecord(初始记录含工具元数据)
    API->>Model: 调用模型(可能返回 tool_use 块 或 流式增量)
    alt 非流式
        Model->>API: 返回完成(可能含 tool_use 或 tool_calls)
        API->>API: extractToolCalls -> ToolCallType[]
        API->>Client: 返回完成(包含 content 与 tool_calls)
    else 流式
        Model->>API: 流式发送增量(含 token 与 tool_use 块)
        API->>API: 跟踪 streamToolCalls 与 toolCallArguments(收集 JSON 参数)
        API->>API: 最终化并附加 tool_calls 到完成记录
        API->>Client: 发送完整流式响应(最终事件包含 tool_calls)
    end
    API->>Logger: 记录最终完成(含工具信息与 tool_calls)
Loading
sequenceDiagram
    participant Browser as 浏览器
    participant DetailPanel as 详情面板
    participant PrettyView as PrettyView 组件
    participant I18n as 国际化模块

    Browser->>DetailPanel: 打开请求详情(含 tool_calls/tools/tool_choice)
    DetailPanel->>PrettyView: 传入请求数据
    PrettyView->>I18n: 读取翻译键(ToolCalls/Tools/ToolChoice 等)
    I18n-->>PrettyView: 返回本地化字符串
    PrettyView->>PrettyView: 渲染消息与响应(显示 tool_calls、NoContent 占位)
    PrettyView->>PrettyView: 渲染 ToolCallDisplay(解析并展示 arguments/id/name)
    PrettyView->>PrettyView: 渲染 ToolsDefinitionDisplay(描述与参数)
    PrettyView->>Browser: 在详情面板中呈现工具相关信息
Loading

预估代码审查工作量

🎯 3 (Moderate) | ⏱️ ~25 minutes

诗歌

🐰 工具呼唤穿林来,参数像萝卜排,

流里藏声与断点,记录把踪迹摆。
前端折叠显细节,徽章点缀小爱,
我在代码菜园里,数着增量唱新菜。

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 拉取请求标题清晰准确地反映了此更改集的主要内容:添加函数调用/工具使用功能及其显示支持。

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello @pescn, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly upgrades the system's ability to handle and visualize OpenAI-compatible function calling and tool use. It introduces new data structures and logic to accurately log all aspects of tool interactions, from initial definitions and choice settings in requests to the actual tool calls made in responses. The corresponding frontend updates provide a clear, interactive display of these details, making it easier to understand how AI models are leveraging external tools.

Highlights

  • Function Calling / Tool Use Support: Implemented comprehensive support for OpenAI-compatible function calling and tool use.
  • Database Logging: Enhanced the completions database to record full tool definitions ('tools'), tool choice settings ('tool_choice'), and tool call responses ('tool_calls').
  • UI Display: Integrated new collapsible UI components ('ToolCallDisplay', 'ToolsDefinitionDisplay') to visualize tool calls and definitions within the request detail panel.
  • Streaming & Non-Streaming Compatibility: Ensured robust extraction and logging of tool calls for both streaming and non-streaming API response formats.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds comprehensive support for OpenAI-compatible function calling and tool use. The changes span both the backend and frontend, including schema updates, API handling for streaming and non-streaming tool calls, and new UI components for displaying tool information.

The implementation is well-done. I've found a bug in the frontend related to parsing tool arguments which could lead to a type error. I've also suggested some refactoring in the backend to improve type safety and reduce code duplication by extracting the tool_choice type definition.

Overall, great work on adding this significant feature.

Comment on lines +471 to +473
parsedArgs = typeof toolCall.function.arguments === 'string'
? JSON.parse(toolCall.function.arguments)
: toolCall.function.arguments
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There's a type mismatch here. The ToolCall interface defines function.arguments as a string. However, the else branch of this ternary operator attempts to assign this string value directly to parsedArgs, which is typed as Record<string, unknown> | null. This will cause a TypeScript compilation error.

Since toolCall.function.arguments is always a string according to its type, the typeof check and the else branch are unnecessary. You can simplify this to just parse the string.

Suggested change
parsedArgs = typeof toolCall.function.arguments === 'string'
? JSON.parse(toolCall.function.arguments)
: toolCall.function.arguments
parsedArgs = JSON.parse(toolCall.function.arguments)

Comment thread backend/src/api/v1/completions.ts Outdated
messages: Array<{ role: string; content: string }>,
messages: CompletionsMessageType[],
tools?: ToolDefinitionType[],
toolChoice?: "auto" | "none" | "required" | { type: "function"; function: { name: string } },
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This complex type for toolChoice is duplicated in a few places. To improve reusability and maintainability, I recommend extracting it into its own exported ToolChoiceType in backend/src/db/schema.ts and then using it here. This will make the code cleaner and easier to maintain.

You'll also need to import ToolChoiceType from @/db/schema.

Suggested change
toolChoice?: "auto" | "none" | "required" | { type: "function"; function: { name: string } },
toolChoice?: ToolChoiceType,

Comment thread backend/src/api/v1/completions.ts Outdated
Comment on lines +536 to +541
const toolChoice = rawBody.tool_choice as
| "auto"
| "none"
| "required"
| { type: "function"; function: { name: string } }
| undefined;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the buildCompletionRecord function, this type assertion can be simplified by using a reusable ToolChoiceType (which I've suggested extracting in backend/src/db/schema.ts). This improves readability and avoids type duplication.

      const toolChoice = rawBody.tool_choice as ToolChoiceType | undefined;

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jan 19, 2026

Note

Docstrings generation - SUCCESS
Generated docstrings for this pull request at #48

coderabbitai Bot added a commit that referenced this pull request Jan 19, 2026
Docstrings generation was requested by @pescn.

* #46 (comment)

The following files were modified:

* `backend/src/api/v1/completions.ts`
* `backend/src/utils/api-helpers.ts`
* `frontend/src/pages/requests/detail-panel/pretty-view.tsx`
- Extract ToolChoiceType to schema.ts for reusability
- Use ToolChoiceType in completions.ts to reduce type duplication
- Simplify JSON.parse in ToolCallDisplay (arguments is always string)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@pescn
Copy link
Copy Markdown
Contributor Author

pescn commented Jan 19, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for OpenAI-compatible function calling and tool use, with changes across the backend and frontend. The backend is updated to log tool definitions, choices, and calls, including robust handling for both streaming and non-streaming responses. The frontend adds new UI components to display this information clearly in the request detail panel. My review identifies a critical issue in the backend's streaming logic that could lead to data corruption and a medium-severity issue on the frontend related to type safety and code clarity. Overall, this is a significant and well-implemented feature, and addressing these points will enhance its robustness.

Comment on lines +311 to 349
if (chunk.type === "content_block_start") {
// Track new tool call block
if (chunk.contentBlock?.type === "tool_use") {
const index = chunk.index ?? 0;
streamToolCalls.set(index, {
id: chunk.contentBlock.id,
type: "function",
function: {
name: chunk.contentBlock.name,
arguments: "",
},
});
toolCallArguments.set(index, []);
}
} else if (chunk.type === "content_block_delta") {
if (chunk.delta?.type === "text_delta" && chunk.delta.text) {
textParts.push(chunk.delta.text);
} else if (
chunk.delta?.type === "thinking_delta" &&
chunk.delta.thinking
) {
thinkingParts.push(chunk.delta.thinking);
} else if (chunk.delta?.type === "input_json_delta" && chunk.delta.partialJson) {
// Collect tool call arguments
const index = chunk.index ?? 0;
const args = toolCallArguments.get(index);
if (args) {
args.push(chunk.delta.partialJson);
}
}
} else if (chunk.type === "content_block_stop") {
// Finalize tool call arguments
const index = chunk.index ?? 0;
const toolCall = streamToolCalls.get(index);
const args = toolCallArguments.get(index);
if (toolCall && args) {
toolCall.function.arguments = args.join("");
}
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The use of chunk.index ?? 0 to get the tool call index can lead to data corruption when processing streamed responses with multiple tool calls. If an adapter fails to provide an index for a tool call chunk (i.e., chunk.index is undefined), it will default to 0. This could cause data for a later tool call to overwrite or mix with data for the first tool call at index 0, resulting in incorrectly logged completion data.

It would be safer to validate that chunk.index is a number when dealing with tool call chunks and log an error if it's missing.

      // Collect content for completion record
      if (chunk.type === "content_block_start") {
        // Track new tool call block
        if (chunk.contentBlock?.type === "tool_use") {
          if (typeof chunk.index !== "number") {
            logger.error("Missing index for tool_use content_block_start in stream");
            continue;
          }
          const index = chunk.index;
          streamToolCalls.set(index, {
            id: chunk.contentBlock.id,
            type: "function",
            function: {
              name: chunk.contentBlock.name,
              arguments: "",
            },
          });
          toolCallArguments.set(index, []);
        }
      } else if (chunk.type === "content_block_delta") {
        if (chunk.delta?.type === "text_delta" && chunk.delta.text) {
          textParts.push(chunk.delta.text);
        } else if (
          chunk.delta?.type === "thinking_delta" &&
          chunk.delta.thinking
        ) {
          thinkingParts.push(chunk.delta.thinking);
        } else if (chunk.delta?.type === "input_json_delta" && chunk.delta.partialJson) {
          // Collect tool call arguments
          if (typeof chunk.index !== "number") {
            logger.error("Missing index for input_json_delta in stream");
            continue;
          }
          const index = chunk.index;
          const args = toolCallArguments.get(index);
          if (args) {
            args.push(chunk.delta.partialJson);
          }
        }
      } else if (chunk.type === "content_block_stop") {
        // Finalize tool call arguments
        if (typeof chunk.index !== "number") {
          logger.error("Missing index for content_block_stop in stream");
          continue;
        }
        const index = chunk.index;
        const toolCall = streamToolCalls.get(index);
        const args = toolCallArguments.get(index);
        if (toolCall && args) {
          toolCall.function.arguments = args.join("");
        }
      }

Comment on lines +125 to +130
// Check if this is a tool result message
const isToolMessage = message.role === 'tool'
const toolCallId = 'tool_call_id' in message ? (message as { tool_call_id?: string }).tool_call_id : undefined

// Check if this is an assistant message with tool calls
const toolCalls = 'tool_calls' in message ? (message as { tool_calls?: ToolCall[] }).tool_calls : undefined
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The property access for tool_call_id and tool_calls is a bit verbose with repeated in checks and type casting. This can be made more concise and readable by casting the message object once to an extended type. This improves maintainability by centralizing the type assertion.

Suggested change
// Check if this is a tool result message
const isToolMessage = message.role === 'tool'
const toolCallId = 'tool_call_id' in message ? (message as { tool_call_id?: string }).tool_call_id : undefined
// Check if this is an assistant message with tool calls
const toolCalls = 'tool_calls' in message ? (message as { tool_calls?: ToolCall[] }).tool_calls : undefined
const extendedMessage = message as RequestMessage & { tool_call_id?: string; tool_calls?: ToolCall[] };
const isToolMessage = extendedMessage.role === 'tool'
const toolCallId = extendedMessage.tool_call_id
const toolCalls = extendedMessage.tool_calls

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/pages/requests/detail-panel/pretty-view.tsx (1)

279-366: 为兼容旧记录,添加从 extraBody 的回退读取

第 279 行的注释说明旧记录可能在 extraBody 中存储 tools/tool_choice,但当前实现仅从 promptData.toolspromptData.tool_choice 读取。建议添加回退读取以确保兼容性:

-  const tools = promptData.tools
-  const toolChoice = promptData.tool_choice
+  const tools =
+    promptData.tools ?? (promptData.extraBody?.tools as ToolDefinition[] | undefined)
+  const toolChoice =
+    promptData.tool_choice ?? (promptData.extraBody?.tool_choice as unknown)
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 070ed11 and 21da30d.

📒 Files selected for processing (3)
  • backend/src/api/v1/completions.ts
  • backend/src/db/schema.ts
  • frontend/src/pages/requests/detail-panel/pretty-view.tsx
🧰 Additional context used
🧬 Code graph analysis (2)
backend/src/api/v1/completions.ts (2)
backend/src/db/schema.ts (4)
  • CompletionsMessageType (105-114)
  • ToolDefinitionType (77-84)
  • ToolChoiceType (89-93)
  • ToolCallType (65-72)
backend/src/utils/api-helpers.ts (2)
  • extractToolCalls (145-164)
  • extractContentText (121-139)
frontend/src/pages/requests/detail-panel/pretty-view.tsx (4)
frontend/src/lib/content.ts (1)
  • extractReasoning (1-13)
frontend/src/components/app/markdown/index.tsx (1)
  • Markdown (11-28)
frontend/src/components/ui/indicator-badge.tsx (1)
  • IndicatorBadge (6-16)
frontend/src/components/ui/collapsible.tsx (3)
  • Collapsible (29-29)
  • CollapsibleTrigger (29-29)
  • CollapsibleContent (29-29)
🔇 Additional comments (13)
frontend/src/pages/requests/detail-panel/pretty-view.tsx (6)

2-11: 图标导入变更合理
新增的图标与工具相关 UI 保持一致,没看到问题。


30-48: 工具相关类型定义清晰
类型结构与 OpenAI 格式一致,便于前端展示与解析。


121-158: 工具消息与工具调用展示处理到位
工具消息标识与工具调用列表的渲染逻辑清晰,交互一致。


162-216: Response 渲染逻辑完整
工具调用区块与无内容占位处理都符合预期。


462-500: 工具调用参数展示健壮
JSON 解析失败回退到原始字符串的处理合理。


503-548: 工具定义展示实现 OK
折叠展示参数与描述的结构清晰、易读。

backend/src/db/schema.ts (1)

62-134: 类型扩展与工具调用格式对齐
新增类型与 prompt/completion 结构扩展合理,便于后续复用与序列化。

backend/src/api/v1/completions.ts (6)

18-29: 类型与工具提取导入一致
新导入的类型与工具调用解析函数与新增逻辑匹配良好。


65-85: 记录结构扩展到 tools/tool_choice 是正确方向
prompt 里补全工具元数据后,日志可追踪性更好。


173-181: 非流式响应补充 tool_calls 正确
从内部响应提取工具调用并写入 completion 的处理合理。


297-349: 流式工具调用追踪实现合理
按 index 聚合 tool_use 与参数片段的逻辑清晰。


369-387: 流式最终 tool_calls 汇总 OK
最终 completion 里补齐 tool_calls 与内容组合逻辑一致。


534-548: 请求体工具元数据抽取用于记录 OK
直接从 rawBody 取 tools/tool_choice 并写入日志,符合记录目的。

✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.

- Fix streaming tool call tracking to use tool ID as key instead of index
  to prevent data corruption when multiple tool calls are processed
- Simplify type access in MessageContent by casting message once
- Add fallback to extraBody for backward compatibility with older records
  that may have tools/tool_choice stored in extraBody

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@pescn
Copy link
Copy Markdown
Contributor Author

pescn commented Jan 19, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for function calling/tool use, a significant feature enhancement. The changes span both the backend and frontend, enabling the logging and display of tool definitions, choices, and calls. The backend modifications correctly update the database schema and API logic for both streaming and non-streaming responses. The frontend adds well-designed collapsible UI components to display this new information.

My review identifies a potential robustness issue in the stream processing logic where a risky fallback could lead to data corruption in certain edge cases. I've provided suggestions to make the implementation more resilient. Overall, this is a solid contribution that greatly expands the gateway's capabilities.

Comment thread backend/src/api/v1/completions.ts Outdated
Comment on lines +341 to +348
const index = chunk.index ?? 0;
const toolId = indexToIdMap.get(index);
if (toolId) {
const args = toolCallArguments.get(toolId);
if (args) {
args.push(chunk.delta.partialJson);
}
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The fallback ?? 0 for chunk.index is risky. If a provider sends a delta for a tool call other than the first one and omits the index, this logic will incorrectly associate the delta with the tool call at index 0. This could lead to corrupted tool call arguments.

While the current adapters might always provide an index, this makes the stream processing logic fragile. It's safer to handle the undefined case explicitly by logging a warning and skipping the chunk. I've also slightly simplified the nested checks.

          const index = chunk.index;
          if (index === undefined) {
            logger.warn("Tool call delta chunk is missing index, skipping.", { chunk });
            continue;
          }
          const toolId = indexToIdMap.get(index);
          const args = toolId ? toolCallArguments.get(toolId) : undefined;
          if (args) {
            args.push(chunk.delta.partialJson);
          }

Comment thread backend/src/api/v1/completions.ts Outdated
Comment on lines 352 to 360
const index = chunk.index ?? 0;
const toolId = indexToIdMap.get(index);
if (toolId) {
const toolCall = streamToolCalls.get(toolId);
const args = toolCallArguments.get(toolId);
if (toolCall && args) {
toolCall.function.arguments = args.join("");
}
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Similar to the content_block_delta case, the fallback ?? 0 for chunk.index is risky here. If an index is missing on a stop chunk for a multi-tool-call response, it could lead to arguments of one tool call not being finalized correctly.

To make the logic more robust, we should explicitly check for an undefined index and handle it gracefully.

        const index = chunk.index;
        if (index === undefined) {
          logger.warn("Tool call stop chunk is missing index, skipping.", { chunk });
          continue;
        }
        const toolId = indexToIdMap.get(index);
        const toolCall = toolId ? streamToolCalls.get(toolId) : undefined;
        const args = toolId ? toolCallArguments.get(toolId) : undefined;
        if (toolCall && args) {
          toolCall.function.arguments = args.join("");
        }

Instead of defaulting to index 0 which could cause data corruption,
now explicitly check if chunk.index is defined before processing
input_json_delta and content_block_stop chunks. Log a warning when
index is missing for input_json_delta.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@pescn
Copy link
Copy Markdown
Contributor Author

pescn commented Jan 19, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is an excellent pull request that adds comprehensive support for function calling and tool use, a significant feature. The changes are well-structured across the backend and frontend, handling both streaming and non-streaming responses, and even considering backward compatibility. The new UI components for displaying tool calls and definitions are a great addition. My main feedback is to improve the request validation schema on the backend to ensure type safety and robustness for the new fields.

Comment thread backend/src/api/v1/completions.ts Outdated
Comment on lines +553 to +561
const rawBody = body as Record<string, unknown>;
const tools = rawBody.tools as ToolDefinitionType[] | undefined;
const toolChoice = rawBody.tool_choice as ToolChoiceType | undefined;

// Build completion record for logging (with full message data)
const completion = buildCompletionRecord(
body.model,
modelConfig.id,
body.messages,
rawBody.messages as CompletionsMessageType[],
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

These type assertions on rawBody are unsafe as they bypass the request validation logic. This is happening because the tChatCompletionCreate schema (defined on line 46) has not been updated to include the new tools, tool_choice fields, or the more complex structure for messages.

To ensure type safety and prevent potential runtime errors from malformed requests, it's crucial to update the tChatCompletionCreate Elysia schema to accurately reflect the expected request body. This would make these type assertions unnecessary and leverage the framework's validation capabilities.

Update tChatCompletionCreate schema to include:
- tToolDefinition: validates tool function definitions
- tToolChoice: validates tool choice (auto/none/required/function)
- tMessage: supports tool_calls, tool_call_id, and name fields

This removes unsafe type assertions on rawBody and leverages
the framework's validation capabilities for type safety.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@pescn
Copy link
Copy Markdown
Contributor Author

pescn commented Jan 19, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds comprehensive support for OpenAI-compatible function calling and tool use. The backend changes correctly handle tool definitions, choices, and calls for both streaming and non-streaming responses, with well-defined database schema and API validation updates. The frontend is also updated to display this new information in a user-friendly, collapsible UI. My review includes a suggestion to improve the robustness of tool argument serialization on the backend and a fix for the frontend to avoid displaying tool information redundantly. Overall, this is a solid implementation of a complex feature.

Comment on lines +152 to +159
toolCalls.push({
id: block.id,
type: "function",
function: {
name: block.name,
arguments: JSON.stringify(block.input),
},
});
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

JSON.stringify can throw an exception if block.input contains circular references. While this is unlikely for tool arguments from an LLM, wrapping this operation in a try-catch block would make the function more robust. This would prevent the entire request from failing due to a single malformed tool call, instead just logging the error and skipping that tool call.

      try {
        const args = JSON.stringify(block.input);
        toolCalls.push({
          id: block.id,
          type: "function",
          function: {
            name: block.name,
            arguments: args,
          },
        });
      } catch (error) {
        logger.error("Failed to stringify tool call arguments for tool_use block", {
          toolName: block.name,
          toolId: block.id,
          error,
        });
      }

Comment on lines 372 to 377
key: 'extraBody',
name: t('pages.requests.detail-panel.pretty-view.ExtraBody'),
value: <ExtraDataDisplay data={data.prompt.extraBody} />,
hidden: !data.prompt.extraBody,
value: <ExtraDataDisplay data={promptData.extraBody} />,
hidden: !promptData.extraBody,
fullWidth: true,
},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To avoid displaying tools and tool_choice in both their dedicated sections and the 'Extra body' section, it's better to filter them out from the extraBody before rendering. This is particularly relevant for older records where these properties might exist inside extraBody. The current implementation could lead to redundant information in the UI.

      key: 'extraBody',
      name: t('pages.requests.detail-panel.pretty-view.ExtraBody'),
      value: (
        <ExtraDataDisplay
          data={
            promptData.extraBody
              ? (({ tools: _t, tool_choice: _tc, ...rest }) => rest)(promptData.extraBody)
              : undefined
          }
        />
      ),
      hidden:
        !promptData.extraBody ||
        Object.keys(promptData.extraBody).filter((k) => k !== 'tools' && k !== 'tool_choice').length === 0,
      fullWidth: true,
    },

@pescn
Copy link
Copy Markdown
Contributor Author

pescn commented Jan 19, 2026

Review Response

Regarding JSON.stringify exception handling (api-helpers.ts:159)

Not adopting this suggestion.

JSON.stringify only throws on circular references, which is impossible for LLM-returned tool arguments (they are standard JSON data). Adding try-catch here would be over-defensive programming that adds unnecessary complexity. If circular references somehow appeared, it would indicate a serious bug in the upstream adapter that should fail fast rather than be silently ignored.

Regarding filtering tools/tool_choice from extraBody (pretty-view.tsx:377)

Not adopting this suggestion.

This suggestion is based on an incorrect assumption that tools might exist in both promptData.tools and promptData.extraBody.tools simultaneously. Since this is a new feature, tools will only be stored in promptData.tools - there are no legacy records with tools in extraBody. Therefore, duplicate display cannot occur.


All other review comments have been addressed. Proceeding to merge.

@pescn pescn merged commit ae4895b into main Jan 19, 2026
2 checks passed
@pescn pescn deleted the feat/function-calling-support branch January 19, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant