fix: improve audio MIME normalization and validation in file reads

[junaiddshaukat]

Summary

Improves audio file handling in the multimodal read path by normalizing common audio MIME aliases (e.g. audio/x-wav → audio/wav), adding extension-based fallback for supported formats (MP3, WAV, AIFF, AAC, OGG, FLAC), and returning a clear early error for unsupported audio instead of failing later with generic API errors. Fixes gaps in fileUtils.ts where raw mime.getType() results were used and unsupported types were not validated.

Details

• Supported audio map: SUPPORTED_AUDIO_MIME_TYPES_BY_EXTENSION maps .mp3, .wav, .aiff, .aif, .aac, .ogg, .flac to canonical MIME types.
• Normalization: AUDIO_MIME_TYPE_NORMALIZATION maps aliases (e.g. audio/x-wav, audio/mp3) to canonical types so inline data sent to the API uses consistent MIME values.
• New helper: getSupportedAudioMimeTypeForFile(filePath) returns the normalized MIME when the file is a supported audio format, otherwise undefined.
• detectFileType: After existing MIME checks, uses extension fallback so supported audio (e.g. .flac) is detected as audio even when MIME lookup is missing.
• processSingleFileContent: For fileType === 'audio', uses getSupportedAudioMimeTypeForFile(). If none, returns early with ToolErrorType.READ_CONTENT_FAILURE and a message listing supported formats. Image/pdf/video unchanged (still use getSpecificMimeType fallback).
• No new dependencies; backward compatible for already-supported audio.

Related Issues

Fixes #21635

How to Validate

1. Run the fileUtils tests:

   npm test -w @google/gemini-cli-core -- src/utils/fileUtils.test.ts
   

All tests should pass, including:

"should detect supported audio files by extension when mime lookup is missing"
"should normalize supported audio mime types before returning inline data"
"should reject unsupported audio mime types with a clear error"

2. Optional manual check: Create a small .wav file and use the read_file tool on it; inline data should have mimeType: 'audio/wav' (or normalized equivalent). Create a file with an unsupported audio extension/MIME (e.g. .adp with audio/adpcm); the tool should return a clear "Unsupported audio file format" error listing supported formats.

3. Edge case: Existing behavior for .adp files with XML content (detected as text per Bug: CLI incorrectly treats .adp files as binary audio (MIME-type mismatch) #16888) is unchanged; only binary audio with unsupported MIME is rejected.

Local Test Results

Pre-Merge Checklist

• Updated relevant documentation and README (if needed)
• Added/updated tests (if needed)
• Noted breaking changes (if any) - None
• Validated on required platforms/methods:
  • MacOS
    • npm run
    • npx
    • Docker
    • Podman
    • Seatbelt
  • Windows
    • npm run
    • npx
    • Docker
  • Linux
    • npm run
    • npx
    • Docker

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the robustness of audio file handling within the multimodal read path. It standardizes audio MIME types, improves file type detection by incorporating extension-based fallbacks, and provides more informative error messages for unsupported audio formats, leading to a more reliable and user-friendly experience.

Highlights

• Audio MIME Normalization: Implemented normalization for common audio MIME aliases (e.g., "audio/x-wav" to "audio/wav") to ensure consistent MIME values for inline data sent to the API.
• Extension-Based Audio Detection: Added extension-based fallback for detecting supported audio formats (MP3, WAV, AIFF, AAC, OGG, FLAC) in detectFileType when MIME lookup is missing.
• Early Error Handling for Unsupported Audio: Introduced clear, early error returns in processSingleFileContent for unsupported audio file formats, preventing later failures with generic API errors.
• New Helper Function: Created getSupportedAudioMimeTypeForFile(filePath) to return the normalized MIME type for supported audio files or undefined otherwise.

Changelog

• packages/core/src/utils/fileUtils.test.ts
  • Imported ToolErrorType for error assertion.
  • Added a test to verify detection of supported audio files by extension when MIME lookup is missing.
  • Included a test to confirm normalization of supported audio MIME types before returning inline data.
  • Added a test to ensure unsupported audio MIME types are rejected with a clear error message.
• packages/core/src/utils/fileUtils.ts
  • Defined SUPPORTED_AUDIO_MIME_TYPES_BY_EXTENSION map for extension-to-MIME mapping.
  • Created AUDIO_MIME_TYPE_NORMALIZATION record to map common audio MIME aliases to canonical types.
  • Implemented getSupportedAudioMimeTypeForFile to determine and normalize the MIME type for audio files.
  • Integrated getSupportedAudioMimeTypeForFile into detectFileType to improve audio file type identification.
  • Modified processSingleFileContent to use the new audio MIME type detection and normalization, and to return a specific error for unsupported audio formats.

Activity

• The author has updated relevant documentation and README (if needed).
• The author has added/updated tests (if needed).
• The author has validated the changes on MacOS using npm run.
• A screenshot of local test results was provided.
• The pull request fixes issue Improve audio MIME normalization and validation in multimodal file reads #21635.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request enhances audio file handling by normalizing MIME types and providing clearer error messages for unsupported formats. A significant security oversight was identified in the processSingleFileContent function, which lacks path validation against the provided rootDirectory, potentially allowing Path Traversal and Denial of Service attacks. This aligns with established repository rules regarding path sanitization and internal validation for utility functions. It is recommended to use the existing isWithinRoot utility to enforce access control. Additionally, to improve maintainability, consider dynamically generating the list of supported audio formats in error messages, avoiding hardcoded strings that could become outdated.

[gemini-cli]

Hi there! Thank you for your interest in contributing to Gemini CLI.

To ensure we maintain high code quality and focus on our prioritized roadmap, we have updated our contribution policy (see Discussion #17383).

We only guarantee review and consideration of pull requests for issues that are explicitly labeled as 'help wanted'. All other community pull requests are subject to closure after 14 days if they do not align with our current focus areas. For this reason, we strongly recommend that contributors only submit pull requests against issues explicitly labeled as 'help-wanted'.

This pull request is being closed as it has been open for 14 days without a 'help wanted' designation. We encourage you to find and contribute to existing 'help wanted' issues in our backlog! Thank you for your understanding and for being part of our community!

[cocosheng-g]

@junaiddshaukat, apologies for the bot closing this PR! We have reopened it. Please sync your branch to the latest main and we will have someone review it shortly.

[cocosheng-g]

/gemini review

[gemini-code-assist]

Code Review

This pull request enhances audio file handling in fileUtils.ts by introducing a robust system for detecting, normalizing, and validating supported audio formats including MP3, WAV, AIFF, AAC, OGG, and FLAC. It includes logic to map various legacy or non-standard MIME types to standard ones and provides clear error reporting for unsupported formats. Corresponding unit tests were added to verify extension-based detection, MIME type normalization, and error handling for unsupported files. I have no feedback to provide.