fix: ollama base64 image error #1207

zerob13 · 2025-12-22T11:48:56Z

Summary by CodeRabbit

Bug Fixes

Fixed image data handling for Ollama provider to properly process different image formats, ensuring images are transmitted correctly.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai · 2025-12-22T11:49:06Z

Walkthrough

The OllamaProvider's formatMessages function now transforms image data during processing. Data URIs (data:image/...;base64,...) are parsed to extract only the base64 payload, while regular URLs remain unchanged. This ensures Ollama receives properly formatted image content.

Changes

Cohort / File(s)	Summary
Image data transformation `src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts`	Modified `formatMessages` to map over image URLs, extracting base64 payload from data URIs while preserving regular URLs unchanged

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Verify the regex or string parsing logic correctly extracts base64 from data URIs
Confirm behavior is correct for both data URIs and regular URL formats

Poem

🐰 A rabbit finds images wrapped tight,
Data URIs shining bright,
Base64 cores are pulled with care,
For Ollama's feast so fair! 📸

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: fixing an issue with base64 image handling in the Ollama provider by extracting base64 payloads from data URIs.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch bugfix/ollama-image

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 079ba05 and 41128d9.

📒 Files selected for processing (1)

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

🧰 Additional context used

📓 Path-based instructions (15)

**/*.{ts,tsx,js,jsx,vue}

📄 CodeRabbit inference engine (CLAUDE.md)

Use English for logs and comments (Chinese text exists in legacy code, but new code should use English)

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

Enable and maintain strict TypeScript type checking for all files

**/*.{ts,tsx}: Always use try-catch to handle possible errors in TypeScript code
Provide meaningful error messages when catching errors
Log detailed error logs including error details, context, and stack traces
Distinguish and handle different error types (UserError, NetworkError, SystemError, BusinessError) with appropriate handlers in TypeScript
Use structured logging with logger.error(), logger.warn(), logger.info(), logger.debug() methods from logging utilities
Do not suppress errors (avoid empty catch blocks or silently ignoring errors)
Provide user-friendly error messages for user-facing errors in TypeScript components
Implement error retry mechanisms for transient failures in TypeScript
Avoid logging sensitive information (passwords, tokens, PII) in logs

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/main/presenter/**/*.ts

📄 CodeRabbit inference engine (CLAUDE.md)

Organize core business logic into dedicated Presenter classes, with one presenter per functional domain

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/main/presenter/llmProviderPresenter/providers/*.ts

📄 CodeRabbit inference engine (CLAUDE.md)

src/main/presenter/llmProviderPresenter/providers/*.ts: Each LLM provider must implement the coreStream method following the standardized event interface for tool calling and response streaming
Convert MCP tools to provider-specific formats and normalize streaming responses to standard events in each provider implementation

src/main/presenter/llmProviderPresenter/providers/*.ts: In Provider implementations (src/main/presenter/llmProviderPresenter/providers/*.ts), the coreStream(messages, modelId, temperature, maxTokens) method should perform a single-pass streaming API request for each conversation round without containing multi-turn tool call loop logic
In Provider implementations, handle native tool support by converting MCP tools to Provider format using convertToProviderTools and including them in the API request; for Providers without native function call support, prepare messages using prepareFunctionCallPrompt before making the API call
In Provider implementations, parse Provider-specific data chunks from the streaming response and yield standardized LLMCoreStreamEvent objects conforming to the standard stream event interface, including text, reasoning, tool calls, usage, errors, stop reasons, and image data
In Provider implementations, include helper methods for Provider-specific operations such as formatMessages, convertToProviderTools, parseFunctionCalls, and prepareFunctionCallPrompt

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/main/**/*.ts

📄 CodeRabbit inference engine (CLAUDE.md)

Use EventBus from src/main/eventbus.ts for main-to-renderer communication, broadcasting events via mainWindow.webContents.send()

src/main/**/*.ts: Use EventBus pattern for inter-process communication within the main process to decouple modules
Use Electron's built-in APIs for file system and native dialogs instead of Node.js or custom implementations

src/main/**/*.ts: Electron main process code belongs in src/main/ with presenters in presenter/ (Window/Tab/Thread/Mcp/Config/LLMProvider) and eventbus.ts for app events
Use the Presenter pattern in the main process for UI coordination

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

**/*.ts

📄 CodeRabbit inference engine (CLAUDE.md)

Do not include AI co-authoring information (e.g., 'Co-Authored-By: Claude') in git commits

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

**/*.{js,ts,jsx,tsx,mjs,cjs}

📄 CodeRabbit inference engine (.cursor/rules/development-setup.mdc)

Write logs and comments in English

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

{src/main/presenter/**/*.ts,src/renderer/**/*.ts}

📄 CodeRabbit inference engine (.cursor/rules/electron-best-practices.mdc)

Implement proper inter-process communication (IPC) patterns using Electron's ipcRenderer and ipcMain APIs

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/main/presenter/llmProviderPresenter/**/*.ts

📄 CodeRabbit inference engine (.cursor/rules/llm-agent-loop.mdc)

Define the standardized LLMCoreStreamEvent interface with fields: type (text | reasoning | tool_call_start | tool_call_chunk | tool_call_end | error | usage | stop | image_data), content (for text), reasoning_content (for reasoning), tool_call_id, tool_call_name, tool_call_arguments_chunk (for streaming), tool_call_arguments_complete (for complete arguments), error_message, usage object with token counts, stop_reason (tool_use | max_tokens | stop_sequence | error | complete), and image_data object with Base64-encoded data and mimeType

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/**/*

📄 CodeRabbit inference engine (.cursor/rules/project-structure.mdc)

New features should be developed in the src directory

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/main/**/*.{js,ts}

📄 CodeRabbit inference engine (.cursor/rules/project-structure.mdc)

Main process code for Electron should be placed in src/main

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/**/*.{ts,tsx,vue,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Use Prettier with single quotes, no semicolons, and 100 character width

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Use OxLint for linting JavaScript and TypeScript files

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/**/*.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

src/**/*.{ts,tsx}: Use camelCase for variable and function names in TypeScript files
Use PascalCase for type and class names in TypeScript
Use SCREAMING_SNAKE_CASE for constant names

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

src/**/*.ts

📄 CodeRabbit inference engine (AGENTS.md)

Use EventBus for inter-process communication events

Files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

🧠 Learnings (10)

📓 Common learnings

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/provider-guidelines.mdc:0-0
Timestamp: 2025-11-25T05:27:39.200Z
Learning: Applies to **/*Provider**/index.ts : Image events: `image_data` event must provide `data` (Base64 encoded) and `mimeType` fields; control single frame size and frequency to avoid blocking

📚 Learning: 2025-11-25T05:27:12.209Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/llm-agent-loop.mdc:0-0
Timestamp: 2025-11-25T05:27:12.209Z
Learning: Applies to src/main/presenter/llmProviderPresenter/providers/*.ts : In Provider implementations, include helper methods for Provider-specific operations such as `formatMessages`, `convertToProviderTools`, `parseFunctionCalls`, and `prepareFunctionCallPrompt`

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

📚 Learning: 2025-11-25T05:26:11.312Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T05:26:11.312Z
Learning: Applies to src/main/presenter/llmProviderPresenter/providers/*.ts : Convert MCP tools to provider-specific formats and normalize streaming responses to standard events in each provider implementation

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

📚 Learning: 2025-11-25T05:27:12.209Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/llm-agent-loop.mdc:0-0
Timestamp: 2025-11-25T05:27:12.209Z
Learning: Applies to src/main/presenter/llmProviderPresenter/providers/*.ts : In Provider implementations, handle native tool support by converting MCP tools to Provider format using `convertToProviderTools` and including them in the API request; for Providers without native function call support, prepare messages using `prepareFunctionCallPrompt` before making the API call

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

📚 Learning: 2025-11-25T05:27:12.209Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/llm-agent-loop.mdc:0-0
Timestamp: 2025-11-25T05:27:12.209Z
Learning: Applies to src/main/presenter/llmProviderPresenter/providers/*.ts : In Provider implementations, parse Provider-specific data chunks from the streaming response and `yield` standardized `LLMCoreStreamEvent` objects conforming to the standard stream event interface, including text, reasoning, tool calls, usage, errors, stop reasons, and image data

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

📚 Learning: 2025-11-25T05:27:12.209Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/llm-agent-loop.mdc:0-0
Timestamp: 2025-11-25T05:27:12.209Z
Learning: Implement separation of concerns where `src/main/presenter/llmProviderPresenter/index.ts` manages the Agent loop and conversation history, while Provider files handle LLM API interactions, Provider-specific request/response formatting, tool definition conversion, and native vs non-native tool call mechanisms

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

📚 Learning: 2025-11-25T05:27:12.209Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/llm-agent-loop.mdc:0-0
Timestamp: 2025-11-25T05:27:12.209Z
Learning: Applies to src/main/presenter/llmProviderPresenter/**/*.ts : Define the standardized `LLMCoreStreamEvent` interface with fields: `type` (text | reasoning | tool_call_start | tool_call_chunk | tool_call_end | error | usage | stop | image_data), `content` (for text), `reasoning_content` (for reasoning), `tool_call_id`, `tool_call_name`, `tool_call_arguments_chunk` (for streaming), `tool_call_arguments_complete` (for complete arguments), `error_message`, `usage` object with token counts, `stop_reason` (tool_use | max_tokens | stop_sequence | error | complete), and `image_data` object with Base64-encoded data and mimeType

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

📚 Learning: 2025-11-25T05:27:39.200Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/provider-guidelines.mdc:0-0
Timestamp: 2025-11-25T05:27:39.200Z
Learning: Applies to **/*Provider**/index.ts : Image events: `image_data` event must provide `data` (Base64 encoded) and `mimeType` fields; control single frame size and frequency to avoid blocking

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

📚 Learning: 2025-11-25T05:27:12.209Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/llm-agent-loop.mdc:0-0
Timestamp: 2025-11-25T05:27:12.209Z
Learning: Applies to src/main/presenter/llmProviderPresenter/providers/*.ts : In Provider implementations (`src/main/presenter/llmProviderPresenter/providers/*.ts`), the `coreStream(messages, modelId, temperature, maxTokens)` method should perform a *single-pass* streaming API request for each conversation round without containing multi-turn tool call loop logic

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

📚 Learning: 2025-11-25T05:27:12.209Z

Learnt from: CR
Repo: ThinkInAIXYZ/deepchat PR: 0
File: .cursor/rules/llm-agent-loop.mdc:0-0
Timestamp: 2025-11-25T05:27:12.209Z
Learning: Applies to src/main/presenter/llmProviderPresenter/index.ts : In `src/main/presenter/llmProviderPresenter/index.ts`, handle `reasoning`, `text`, `image_data`, and `usage` events by processing and forwarding them through `STREAM_EVENTS.RESPONSE` events to the frontend

Applied to files:

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: build-check (x64)

🔇 Additional comments (1)

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts (1)

122-130: Implementation correctly handles Ollama's base64 image format requirement.

The code properly extracts raw base64 strings from data URIs, which aligns with Ollama's requirement that images be provided as an array with individual images encoded in Base64. Data URIs should not be sent directly to the Ollama API.

coderabbitai · 2025-12-22T11:51:54Z

src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts

+        // Extract base64 data from data URIs (Ollama expects just the base64 string, not the full data URI)
+        const images: string[] = rawImages.map((imgUrl) => {
+          // If it's a data URI (data:image/...;base64,...), extract just the base64 part
+          if (imgUrl.startsWith('data:image') && imgUrl.includes('base64,')) {
+            return imgUrl.split(',')[1]
+          }
+          // For regular URLs, pass as-is
+          return imgUrl
+        })


⚠️ Potential issue | 🟠 Major

Add error handling for malformed data URIs.

The transformation logic assumes well-formed data URIs but lacks defensive checks. If a data URI is malformed (e.g., missing the comma separator or having an unexpected format), imgUrl.split(',')[1] could return undefined, potentially causing issues downstream when Ollama processes the images array.

As per coding guidelines, TypeScript code should handle possible errors with try-catch blocks and provide meaningful error messages.

🔎 Proposed fix with error handling

- // Extract base64 data from data URIs (Ollama expects just the base64 string, not the full data URI) - const images: string[] = rawImages.map((imgUrl) => { - // If it's a data URI (data:image/...;base64,...), extract just the base64 part - if (imgUrl.startsWith('data:image') && imgUrl.includes('base64,')) { - return imgUrl.split(',')[1] - } - // For regular URLs, pass as-is - return imgUrl - }) + // Extract base64 data from data URIs (Ollama expects just the base64 string, not the full data URI) + const images: string[] = rawImages + .map((imgUrl) => { + try { + // If it's a data URI (data:image/...;base64,...), extract just the base64 part + if (imgUrl.startsWith('data:image') && imgUrl.includes('base64,')) { + const parts = imgUrl.split(',') + if (parts.length < 2 || !parts[1]) { + console.warn('Malformed data URI, missing base64 data:', imgUrl.substring(0, 50)) + return null + } + return parts[1] + } + // For regular URLs, pass as-is + return imgUrl + } catch (error) { + console.error('Error processing image URL:', error) + return null + } + }) + .filter((img): img is string => img !== null)

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

// Extract base64 data from data URIs (Ollama expects just the base64 string, not the full data URI)

const images: string[] = rawImages.map((imgUrl) => {

// If it's a data URI (data:image/...;base64,...), extract just the base64 part

if (imgUrl.startsWith('data:image') && imgUrl.includes('base64,')) {

return imgUrl.split(',')[1]

}

// For regular URLs, pass as-is

return imgUrl

})

// Extract base64 data from data URIs (Ollama expects just the base64 string, not the full data URI)

const images: string[] = rawImages

.map((imgUrl) => {

try {

// If it's a data URI (data:image/...;base64,...), extract just the base64 part

if (imgUrl.startsWith('data:image') && imgUrl.includes('base64,')) {

const parts = imgUrl.split(',')

if (parts.length < 2 || !parts[1]) {

console.warn('Malformed data URI, missing base64 data:', imgUrl.substring(0, 50))

return null

}

return parts[1]

}

// For regular URLs, pass as-is

return imgUrl

} catch (error) {

console.error('Error processing image URL:', error)

return null

}

})

.filter((img): img is string => img !== null)

🤖 Prompt for AI Agents

In src/main/presenter/llmProviderPresenter/providers/ollamaProvider.ts around lines 122 to 130, the data-URI handling assumes a well-formed "data:image...;base64,..." string and directly accesses split(',')[1], which can be undefined for malformed URIs; wrap the transformation in a try-catch, validate that imgUrl includes both "data:image" and "base64," and that split(',')[1] exists (not undefined/empty); if validation fails, either throw a descriptive Error or log and skip/return a safe fallback for that item so the images array contains only valid base64 strings (or clear errors) before passing to Ollama.

* refactor: extract agent mode and workspace for every llm * feat: support workdir on agent mode * feat: add i18n * chore: update doc * feat: add support for yo browser in agent look * feat: auto refresh file list when file tool called * chore: acp agent support file event * feat: move mode switch to left * feat: change icon position * chore: upgrade markstream-vue Performance optimizations allow rendering of over 100,000 content items at once. * fix(style): pollution global style * fix: ollama base64 image error (#1207) * fix: remove cursor debug log * feat: add support for workspace with artifacts * feat: merge two workspace components into one * fix: remove cursor debug log * fix: remove style * fix: workspace agent regex and few ai review issues * fix: add safe check for regex patterns * fix: remove deprecated code * chore: use logger to replace console * fix: add error handle for tool call * fix: first time open browser stop loop * fix: log and no blocking async * fix: add browser position * fix: fix renderer event loop * fix: tool call activateTab should not focus browser window * feat: disable preload for browser window * fix: remove cursor debug fetch code --------- Co-authored-by: Simon He <674949287@qq.com>

fix: ollama base64 image error

41128d9

coderabbitai bot reviewed Dec 22, 2025

View reviewed changes

zerob13 merged commit 34ea69e into dev Dec 22, 2025
2 checks passed

zerob13 added a commit that referenced this pull request Dec 22, 2025

fix: ollama base64 image error (#1207)

727eaca

zerob13 mentioned this pull request Dec 22, 2025

[BUG]illegal base64 data at input byte 4 #525

Closed

zerob13 deleted the bugfix/ollama-image branch January 6, 2026 12:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: ollama base64 image error #1207

fix: ollama base64 image error #1207

Uh oh!

zerob13 commented Dec 22, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Dec 22, 2025 •

edited

Loading

Uh oh!

coderabbitai bot left a comment

Uh oh!

coderabbitai bot Dec 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fix: ollama base64 image error #1207

fix: ollama base64 image error #1207

Uh oh!

Conversation

zerob13 commented Dec 22, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Bug Fixes

Uh oh!

coderabbitai bot commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

zerob13 commented Dec 22, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Dec 22, 2025 •

edited

Loading