Skip to content

fix(cli): make yargs extension tests language-independent#8242

Closed
amsminn wants to merge 1 commit intogoogle-gemini:mainfrom
amsminn:fix/test-locale-issues
Closed

fix(cli): make yargs extension tests language-independent#8242
amsminn wants to merge 1 commit intogoogle-gemini:mainfrom
amsminn:fix/test-locale-issues

Conversation

@amsminn
Copy link
Copy Markdown
Contributor

@amsminn amsminn commented Sep 11, 2025

TLDR

Fix locale-dependent test failures in yargs extension command tests by removing hardcoded
English error messages and verifying error occurrence instead of specific message content.

Dive Deeper

The yargs library localizes error messages based on system locale. The existing tests were
hardcoded to expect English error messages, causing failures on non-English systems (e.g.,
Korean). Changed assertions from .toThrow('specific message') to .toThrow() to make tests
language-agnostic.

Reviewer Test Plan

  1. Reproduce the issue (optional):
# Set your system to a non-English locale
export LANG=ko_KR.UTF-8  # or any non-English locale

# Run tests
  1. Verify the fix:
# Checkout this branch
git checkout fix/test-locale-issues

# Run tests

Testing Matrix

🍏 🪟 🐧
npm run
npx
Docker
Podman
Seatbelt - -

Linked issues / bugs

This PR makes progress on #8241

@amsminn amsminn requested a review from a team as a code owner September 11, 2025 02:30
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @amsminn, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where CLI tests were failing on systems configured with non-English locales. By enforcing an English locale for yargs within the test setup, the PR ensures consistent test execution regardless of the host system's language settings, thereby enhancing the reliability and portability of the test suite.

Highlights

  • Test Stability: Resolved test failures occurring on non-English systems by explicitly setting the locale for yargs within the CLI test environment. This ensures yargs error messages are consistently in English, preventing mismatches with test expectations.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses an issue with test failures on non-English systems by forcing an English locale within the test setup. The method of setting environment variables is appropriate for this purpose. My review focuses on a small improvement to remove a redundant environment variable setting, which will enhance the clarity and maintainability of the configuration.

@cornmander
Copy link
Copy Markdown
Contributor

Thank you for your contribution. I think this is an OK way to do this but I would rather that our tests don't make assumptions that break internationalization. We're currently working with a contributor on landing internationalization in #8060 so it would be counterproductive to hardcode our tests to English.

Do you have a list of tests that are broken, and can we make them do less literal string comparison to make them pass?

@amsminn amsminn force-pushed the fix/test-locale-issues branch from 6c2c980 to 3cac454 Compare September 12, 2025 08:16
@amsminn amsminn changed the title fix(cli): force English locale for yargs in tests fix(cli): make yargs extension tests language-independent Sep 12, 2025
@amsminn
Copy link
Copy Markdown
Contributor Author

amsminn commented Sep 12, 2025

Do you have a list of tests that are broken, and can we make them do less literal string comparison to make them pass?

Thank you for the feedback!
The PR work was updated from fixing the test language via env variables to removing language-dependent literal string comparisons altogether.

Copy link
Copy Markdown
Contributor

@cornmander cornmander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@amsminn
Copy link
Copy Markdown
Contributor Author

amsminn commented Sep 12, 2025

The only failing Windows test, SettingsDialog.test.tsx, in Test (Windows-latest, 24.x) doesn’t seem related to my changes.

The failing test should toggle setting with Enter key depends on JavaScript object key ordering, which varies between platforms (Windows vs macOS/Linux).

Can anyone confirm this issue?

@gemini-code-assist

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Thank you for the update and for tagging me, @amsminn. I can confirm that the SettingsDialog.test.tsx file is indeed outside the scope of the changes in this pull request. Your modifications are focused on install.test.ts, new.test.ts, and uninstall.test.ts to address language-dependent string comparisons.

Regarding the "JavaScript object key ordering" issue, while modern JavaScript environments generally preserve insertion order for object keys, relying on this implicitly in tests can sometimes lead to unexpected behavior across different platforms or Node.js versions if the order is critical for the test's assertion. If the test is failing due to an assumed order that isn't explicitly enforced (e.g., by sorting keys before comparison), it could certainly manifest as a platform-specific issue.

Given that this test failure appears unrelated to your current changes and seems to highlight a pre-existing issue with the SettingsDialog.test.tsx test itself, it would likely be best to investigate this in a separate issue or pull request. Your current changes effectively address the language-dependency problem as intended for the yargs extension tests.

@gemini-cli gemini-cli bot added priority/p1 Important and should be addressed in the near term. area/core Issues related to User Interface, OS Support, Core Functionality sub-area/i18n and removed status/possible-duplicate labels Sep 12, 2025
@amsminn amsminn requested a review from a team as a code owner September 13, 2025 02:39
@cornmander cornmander added this pull request to the merge queue Sep 13, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 13, 2025
@amsminn
Copy link
Copy Markdown
Contributor Author

amsminn commented Sep 13, 2025

The only failing Windows test, SettingsDialog.test.tsx, in Test (Windows-latest, 24.x) doesn’t seem related to my changes.
The failing test should toggle setting with Enter key depends on JavaScript object key ordering, which varies between platforms (Windows vs macOS/Linux).
Can anyone confirm this issue?
@gemini-code-assist

@cornmander
The Test (Windows-latest, 22.x) failure that occurred in the merge queue doesn’t seem to be caused by my PR. Could you please check?

@amsminn amsminn force-pushed the fix/test-locale-issues branch from 275d277 to 019d9b6 Compare September 22, 2025 01:42
@gsehgal gsehgal self-requested a review October 22, 2025 08:50
Copy link
Copy Markdown
Contributor

@gsehgal gsehgal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amsminn please squash the commits.

@gsehgal gsehgal self-assigned this Oct 22, 2025
gsehgal

This comment was marked as resolved.

@amsminn amsminn force-pushed the fix/test-locale-issues branch from b44bd09 to 6a4c53f Compare October 22, 2025 14:22
@amsminn
Copy link
Copy Markdown
Contributor Author

amsminn commented Oct 22, 2025

@amsminn please squash the commits.

Done! Commits squashed. Thanks for reviewing

@amsminn amsminn force-pushed the fix/test-locale-issues branch 4 times, most recently from 5c7cf94 to 1852cc0 Compare October 25, 2025 04:59
@jacob314
Copy link
Copy Markdown
Contributor

jacob314 commented Jan 7, 2026

closing in favor of
#14496 (review)
that fixes this issue by adding LC_ALL=C

Something went wrong with merging this PR resulting in an enormous diff.

@jacob314 jacob314 closed this Jan 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Issues related to User Interface, OS Support, Core Functionality priority/p1 Important and should be addressed in the near term.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants