Skip to content

fix: decode Uint8Array and multi-byte UTF-8 in API error messages#23341

Merged
cocosheng-g merged 5 commits intogoogle-gemini:mainfrom
kimjune01:fix/19851-decode-error-bytes
Apr 10, 2026
Merged

fix: decode Uint8Array and multi-byte UTF-8 in API error messages#23341
cocosheng-g merged 5 commits intogoogle-gemini:mainfrom
kimjune01:fix/19851-decode-error-bytes

Conversation

@kimjune01
Copy link
Copy Markdown
Contributor

Summary

Fixes #19851

  • _fixGaxiosErrorData() now handles Uint8Array/Buffer instances directly (not just comma-separated strings)
  • Replaced String.fromCharCode() with TextDecoder for correct multi-byte UTF-8 decoding
  • Added byte range validation (0-255) to avoid false positives on comma-separated strings that aren't byte values

Test plan

  • Existing tests pass (32/32)
  • New test: Uint8Array data is decoded to string
  • New test: multi-byte UTF-8 comma-separated bytes (e.g. "72,195,169,108,108,111""Héllo")
  • New test: values outside 0-255 are left unchanged
  • Existing test: non-numeric comma strings are left unchanged

…ogle-gemini#19851)

Gaxios may return error response bodies as Uint8Array objects or
comma-separated byte strings. The existing workaround only handled
the string case and used String.fromCharCode which breaks on multi-byte
UTF-8. Switch to TextDecoder for correct UTF-8 decoding, and add a
branch for direct Uint8Array/Buffer instances.
@kimjune01 kimjune01 requested a review from a team as a code owner March 21, 2026 05:30
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the error handling mechanism by improving the decoding of API error messages originating from Gaxios. It addresses issues where error response data might be returned as raw byte arrays or comma-separated byte strings, ensuring that these messages are correctly interpreted and displayed as human-readable UTF-8 strings, thereby improving the clarity and utility of error logging.

Highlights

  • Direct Uint8Array Handling: The _fixGaxiosErrorData() function now directly processes Uint8Array and Buffer instances, ensuring proper decoding of raw byte data from Gaxios errors.
  • Multi-byte UTF-8 Decoding: Replaced String.fromCharCode() with TextDecoder to correctly handle multi-byte UTF-8 characters when decoding comma-separated byte strings.
  • Byte Range Validation: Implemented validation to ensure that comma-separated strings are only treated as byte arrays if all values are within the 0-255 range, preventing incorrect decoding of non-byte strings.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Cherry-picked from blind codex review — rejects non-integer values
like 1.5 that pass the isNaN check.
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively addresses a bug in how Gaxios error messages are decoded. The implementation correctly handles Uint8Array and multi-byte UTF-8 strings by using TextDecoder, which is a significant improvement over the previous String.fromCharCode() approach. The addition of byte-range validation for comma-separated strings adds robustness. The new tests are comprehensive and validate the fix. The changes are well-implemented and I have no further recommendations.

Covers Japanese characters (こんにちは) where each char is 3 UTF-8 bytes.
Complements existing 2-byte test (Héllo). From codex blind review.
@gemini-cli gemini-cli Bot added priority/p1 Important and should be addressed in the near term. area/core Issues related to User Interface, OS Support, Core Functionality help wanted We will accept PRs from all issues marked as "help wanted". Thanks for your support! labels Mar 21, 2026
@cocosheng-g
Copy link
Copy Markdown
Contributor

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the _fixGaxiosErrorData method in loggingContentGenerator.ts to correctly decode Gaxios error responses that may arrive as Uint8Array or comma-separated byte strings, ensuring proper UTF-8 handling. Corresponding unit tests have been added to loggingContentGenerator.test.ts to verify these decoding scenarios and handle invalid byte ranges. I have no feedback to provide as there are no review comments.

@cocosheng-g cocosheng-g enabled auto-merge April 10, 2026 00:01
@cocosheng-g cocosheng-g added this pull request to the merge queue Apr 10, 2026
Merged via the queue into google-gemini:main with commit 69bf2d7 Apr 10, 2026
27 checks passed
ik-gemini-bot added a commit to spigell/gemini-cli that referenced this pull request Apr 23, 2026
* refactor(plan): simplify policy priorities and consolidate read-only rules (google-gemini#24849)

* feat(test-utils): add memory usage integration test harness (google-gemini#24876)

* feat(memory): add /memory inbox command for reviewing extracted skills (google-gemini#24544)

* chore(release): bump version to 0.39.0-nightly.20260408.e77b22e63 (google-gemini#24939)

* fix(core): ensure robust sandbox cleanup in all process execution paths (google-gemini#24763)

Co-authored-by: Spencer <spencertang@google.com>

* chore: update ink version to 6.6.8 (google-gemini#24934)

* Changelog for v0.38.0-preview.0 (google-gemini#24938)

Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com>
Co-authored-by: g-samroberts <samroberts@google.com>

* chore: ignore conductor directory (google-gemini#22128)

Co-authored-by: Coco Sheng <cocosheng@google.com>

* Changelog for v0.37.0 (google-gemini#24940)

Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com>
Co-authored-by: Sam Roberts <158088236+g-samroberts@users.noreply.github.com>

* feat(plan): require user confirmation for activate_skill in Plan Mode (google-gemini#24946)

* feat(test-utils): add CPU performance integration test harness (google-gemini#24951)

* fix(core): resolve windows symlink bypass and stabilize sandbox integration tests (google-gemini#24834)

* test(sdk): add unit tests for GeminiCliSession (google-gemini#21897)

* fix(cli): restore file path display in edit and write tool confirmations (google-gemini#24974)

* fix(cli-ui): enable Ctrl+Backspace for word deletion in Windows Terminal (google-gemini#21447)

* fix(core): dynamic session ID injection to resolve resume bugs (google-gemini#24972)

* Update ink version to 6.6.9 (google-gemini#24980)

* feat(core): refine shell tool description display logic (google-gemini#24903)

* Generalize evals infra to support more types of evals, organization and queuing of named suites (google-gemini#24941)

* fix(cli): optimize startup with lightweight parent process (google-gemini#24667)

* refactor(sandbox): use centralized sandbox paths in macOS Seatbelt implementation (google-gemini#24984)

* feat(cli): refine tool output formatting for compact mode (google-gemini#24677)

* fix(sdk): skip broken sendStream tests to unblock nightly (google-gemini#25000)

* refactor(core): use centralized path resolution for Linux sandbox (google-gemini#24985)

* Support ctrl+shift+g (google-gemini#25035)

* feat(core): refactor subagent tool to unified invoke_subagent tool (google-gemini#24489)

* fix(core): add explicit git identity env vars to prevent sandbox checkpointing error (google-gemini#19775)

Co-authored-by: David Pierce <davidapierce@google.com>

* fix: respect hideContextPercentage when FooterConfigDialog is closed without changes (google-gemini#24773)

Co-authored-by: Coco Sheng <cocosheng@google.com>

* fix(cli): suppress unhandled AbortError logs during request cancellation (google-gemini#22621)

* Automated documentation audit (google-gemini#24567)

* feat(cli): implement useAgentStream hook (google-gemini#24292)

Co-authored-by: Adam Weidman <adamfweidman@gmail.com>
Co-authored-by: Adam Weidman <adamfweidman@google.com>

* refactor(core): remove legacy subagent wrapping tools (google-gemini#25053)

* refactor(plan) Clean default plan toml (google-gemini#25037)

* fix(core): honor retryDelay in RetryInfo for 503 errors (google-gemini#25057)

* fix(core): remediate subagent memory leaks using AbortSignal in MessageBus (google-gemini#25048)

* feat(cli): wire up useAgentStream in AppContainer (google-gemini#24297)

Co-authored-by: Adam Weidman <adamfweidman@gmail.com>
Co-authored-by: Adam Weidman <adamfweidman@google.com>

* feat(core): migrate chat recording to JSONL streaming (google-gemini#23749)

* fix(core): clear 5-minute timeouts in oauth flow to prevent memory leaks (google-gemini#24968)

* fix(sandbox): centralize async git worktree resolution and enforce read-only security (google-gemini#25040)

* feat(test): add high-volume shell test and refine perf harness (google-gemini#24983)

* fix(core): silently handle EPERM when listing dir structure (google-gemini#25066)

* Changelog for v0.37.1 (google-gemini#25055)

Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com>

* fix: decode Uint8Array and multi-byte UTF-8 in API error messages (google-gemini#23341)

Co-authored-by: Coco Sheng <cocosheng@google.com>

* Automated documentation audit results (google-gemini#22755)

* debugging(ui): add optional debugRainbow setting (google-gemini#25088)

* fix: resolve lifecycle memory leaks by cleaning up listeners and root closures (google-gemini#25049)

* docs(cli): updates f12 description to be more precise (google-gemini#15816)

* fix(cli): mark /settings as unsafe to run concurrently (google-gemini#25061)

* fix(core): remove buffer slice to prevent OOM on large output streams (google-gemini#25094)

* feat(core): persist subagent agentId in tool call records (google-gemini#25092)

* chore(core): increase codebase investigator turn limits to 50 (google-gemini#25125)

* refactor(core): consolidate execute() arguments into ExecuteOptions (google-gemini#25101)

* feat(core): add Strategic Re-evaluation guidance to system prompt (google-gemini#25062)

* fix(core): preserve shell execution config fields on update (google-gemini#25113)

* docs: add vi shortcuts and clarify MCP sandbox setup (google-gemini#21679)

Co-authored-by: Jenna Inouye <jinouye@google.com>

* fix(cli): pass session id to interactive shell executions (google-gemini#25114)

* fix(cli): resolve text sanitization data loss due to C1 control characters (google-gemini#22624)

* feat(core): add large memory regression test (google-gemini#25059)

* fix(core): resolve PTY exhaustion and orphan MCP subprocess leaks (google-gemini#25079)

* chore: switch from keytar to @github/keytar (google-gemini#25143)

* chore(deps): update vulnerable dependencies via npm audit fix (google-gemini#25140)

* perf(sandbox): optimize Windows sandbox initialization via native ACL application (google-gemini#25077)

* fix: improve audio MIME normalization and validation in file reads (google-gemini#21636)

Co-authored-by: Coco Sheng <cocosheng@google.com>

* docs: Update docs-audit to include changes in PR body (google-gemini#25153)

* docs: correct documentation for enforced authentication type (google-gemini#25142)

* fix(cli): exclude update_topic from confirmation queue count (google-gemini#24945)

* Memory fix for trace's streamWrapper. (google-gemini#25089)

* fix(core): fix quota footer for non-auto models and improve display (google-gemini#25121)

* docs(contributing): clarify self-assignment policy for issues (google-gemini#23087)

* feat(core): add skill patching support with /memory inbox integration (google-gemini#25148)

* Stop suppressing thoughts and text in model response (google-gemini#25073)

* fix(release): prefix git hash in nightly versions to prevent semver normalization (google-gemini#25304)

* feat(cli): extract QuotaContext and resolve infinite render loop (google-gemini#24959)

* refactor(core): extract and centralize sandbox path utilities (google-gemini#25305)

Co-authored-by: David Pierce <davidapierce@google.com>

* feat(ui): added enhancements to scroll momentum (google-gemini#24447)

* fix(core): replace custom binary detection with isbinaryfile to correctly handle UTF-8 (U+FFFD) (google-gemini#25297)

* feat(agent): implement tool-controlled display protocol (Steps 2-3) (google-gemini#25134)

* Stop showing scrollbar unless we are in terminalBuffer mode (google-gemini#25320)

* fix(core): expose GEMINI_PLANS_DIR to hook environment (google-gemini#25296)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* feat: support auth block in MCP servers config in agents (google-gemini#24770)

* feat(core): implement silent fallback for Plan Mode model routing (google-gemini#25317)

* fix: correct redirect count increment in fetchJson (google-gemini#24896)

Co-authored-by: Tommaso Sciortino <sciortino@gmail.com>

* fix(core): prevent secondary crash in ModelRouterService finally block (google-gemini#25333)

* feat(core): introduce decoupled ContextManager and Sidecar architecture (google-gemini#24752)

* docs(core): update generalist agent documentation (google-gemini#25325)

* chore(mcp): check MCP error code over brittle string match (google-gemini#25381)

* test(core): improve sandbox integration test coverage and fix OS-specific failures (google-gemini#25307)

Co-authored-by: David Pierce <davidapierce@google.com>

* feat(plan): update plan mode prompt to allow showing plan content (google-gemini#25058)

* fix(core): use debug level for keychain fallback logging (google-gemini#25398)

* feat(test): add a performance test in asian language (google-gemini#25392)

* feat(cli): enable mouse clicking for cursor positioning in AskUser multi-line answers (google-gemini#24630)

* fix(core): detect kmscon terminal as supporting true color (google-gemini#25282)

Co-authored-by: Adib234 <30782825+Adib234@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* ci: add agent session drift check workflow (google-gemini#25389)

* use macos-latest-large runner where applicable. (google-gemini#25413)

* Changelog for v0.37.2 (google-gemini#25336)

Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com>

* chore(release): v0.39.0-preview.0

* fix(patch): cherry-pick a4e98c0 to release/v0.39.0-preview.0-pr-25138 to patch version v0.39.0-preview.0 and create version 0.39.0-preview.1 (google-gemini#25766)

Co-authored-by: Mahima Shanware <mahima.shanware@gmail.com>

* chore(release): v0.39.0-preview.1

* fix(patch): cherry-pick d6f88f8 to release/v0.39.0-preview.1-pr-25670 to patch version v0.39.0-preview.1 and create version 0.39.0-preview.2 (google-gemini#25776)

Co-authored-by: Adam Weidman <65992621+adamfweidman@users.noreply.github.com>

* chore(release): v0.39.0-preview.2

* chore(release): v0.39.0

---------

Co-authored-by: ruomeng <ruomeng@google.com>
Co-authored-by: Sri Pasumarthi <111310667+sripasg@users.noreply.github.com>
Co-authored-by: Sandy Tao <sandytao520@icloud.com>
Co-authored-by: gemini-cli-robot <gemini-cli-robot@google.com>
Co-authored-by: Emily Hedlund <ehedlund@google.com>
Co-authored-by: Spencer <spencertang@google.com>
Co-authored-by: Jacob Richman <jacob314@gmail.com>
Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com>
Co-authored-by: g-samroberts <samroberts@google.com>
Co-authored-by: JAYADITYA <96861162+JayadityaGit@users.noreply.github.com>
Co-authored-by: Coco Sheng <cocosheng@google.com>
Co-authored-by: Sam Roberts <158088236+g-samroberts@users.noreply.github.com>
Co-authored-by: Adamya Singh <adamyasingh54@gmail.com>
Co-authored-by: Jarrod Whelan <150866123+jwhelangoog@users.noreply.github.com>
Co-authored-by: dogukanozen <dogukannozen@hotmail.com>
Co-authored-by: Tommaso Sciortino <sciortino@gmail.com>
Co-authored-by: Christian Gunderman <gundermanc@google.com>
Co-authored-by: Sehoon Shon <sshon@google.com>
Co-authored-by: Abhi <43648792+abhipatel12@users.noreply.github.com>
Co-authored-by: MD. MOHIBUR RAHMAN <35300157+mrpmohiburrahman@users.noreply.github.com>
Co-authored-by: David Pierce <davidapierce@google.com>
Co-authored-by: chernistry <73943355+chernistry@users.noreply.github.com>
Co-authored-by: euxaristia <25621994+euxaristia@users.noreply.github.com>
Co-authored-by: Michael Bleigh <mbleigh@mbleigh.com>
Co-authored-by: Adam Weidman <adamfweidman@gmail.com>
Co-authored-by: Adam Weidman <adamfweidman@google.com>
Co-authored-by: Yuna Seol <yunaseol@gmail.com>
Co-authored-by: June <kimjune01@gmail.com>
Co-authored-by: Aishanee Shah <aishaneeshah@google.com>
Co-authored-by: Jason Matthew Suhari <jasonmatthewsuhari@gmail.com>
Co-authored-by: Christopher Thomas <cobekgn@gmail.com>
Co-authored-by: Jenna Inouye <jinouye@google.com>
Co-authored-by: cynthialong0-0 <82900738+cynthialong0-0@users.noreply.github.com>
Co-authored-by: M Junaid Shaukat <154750865+junaiddshaukat@users.noreply.github.com>
Co-authored-by: Abhijit Balaji <abhijitbalaji@google.com>
Co-authored-by: Mark Griffith <anthraxmilkshake@hotmail.com>
Co-authored-by: Jack Wotherspoon <jackwoth@google.com>
Co-authored-by: Jesse Rosenstock <jesse.rosenstock@gmail.com>
Co-authored-by: Adib234 <30782825+Adib234@users.noreply.github.com>
Co-authored-by: Dev Randalpura <devrandalpura@google.com>
Co-authored-by: Anjaligarhwal <anjaligarhwal1610@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Tanmay Vartak <9002434+TanmayVartak@users.noreply.github.com>
Co-authored-by: Jerop Kipruto <jerop@google.com>
Co-authored-by: Kevin Zhao <kevin8093@126.com>
Co-authored-by: joshualitt <joshualitt@google.com>
Co-authored-by: Clay <claygeo6@gmail.com>
Co-authored-by: Adam Weidman <65992621+adamfweidman@users.noreply.github.com>
Co-authored-by: Mahima Shanware <mahima.shanware@gmail.com>
Co-authored-by: codex-bot <spigelly+gh-bot@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Issues related to User Interface, OS Support, Core Functionality help wanted We will accept PRs from all issues marked as "help wanted". Thanks for your support! priority/p1 Important and should be addressed in the near term.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API error messages displayed as raw byte codes instead of decoded text when model capacity is exhausted

2 participants