feat(core): add large memory regression test

[cynthialong0-0]

Summary

Implements memory regression tests for cases:

1. a large chat
2. resuming a large chat
3. entering new messages after resuming a large chat

also track object references in the memory usage

Details

Added 3 tests under memory-usage.test.ts for large chat scenarios mentioned above.

Added large chat test conversation file large-chat-session.json which hold ~1400 messages ~60MB

Added externalBytes in memoryBaseline for tracking the object reference in the memory usage.

Related Issues

Fixes #24869

How to Validate

run npm run test:memory

Pre-Merge Checklist

• Updated relevant documentation and README (if needed)
• Added/updated tests (if needed)
• Noted breaking changes (if any)
• Validated on required platforms/methods:
  • MacOS
    • npm run
    • npx
    • Docker
    • Podman
    • Seatbelt
  • Windows
    • npm run
    • npx
    • Docker
  • Linux
    • npm run
    • npx
    • Docker

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces comprehensive memory regression testing for large chat scenarios to ensure stability and performance under heavy load. It enhances the existing testing harness to track external memory usage and updates the data generation utilities to handle large payloads efficiently, addressing a specific memory-related issue.

Highlights

• Memory Regression Testing: Added three new memory regression test scenarios covering large chat interactions, including initial chat, resuming, and appending messages.
• External Memory Tracking: Updated the memory testing infrastructure to track and baseline 'externalBytes' in addition to heap and RSS metrics.
• Large File Handling: Refactored fakeContentGenerator to use streaming for reading large response files, preventing memory issues during test setup.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

Size Change: -4 B (0%)

Total Size: 34 MB

Filename	Size	Change
./bundle/chunk-3WCA3BPO.js	0 B	-14.8 MB (removed)	🏆
./bundle/chunk-4YKEPKUR.js	0 B	-3.16 MB (removed)	🏆
./bundle/chunk-KZT57HIN.js	0 B	-3.8 kB (removed)	🏆
./bundle/core-YSFZYEN5.js	0 B	-46 kB (removed)	🏆
./bundle/devtoolsService-TJ6RSL3L.js	0 B	-28.4 kB (removed)	🏆
./bundle/gemini-TW4QDMIY.js	0 B	-552 kB (removed)	🏆
./bundle/interactiveCli-HOAJIGAM.js	0 B	-1.66 MB (removed)	🏆
./bundle/oauth2-provider-KWJ375BG.js	0 B	-9.16 kB (removed)	🏆
./bundle/chunk-OEGGAO7X.js	14.8 MB	+14.8 MB (new file)	🆕
./bundle/chunk-PJQOL4QZ.js	3.8 kB	+3.8 kB (new file)	🆕
./bundle/chunk-UXJQ7HQO.js	3.16 MB	+3.16 MB (new file)	🆕
./bundle/core-TTA3YEDX.js	46 kB	+46 kB (new file)	🆕
./bundle/devtoolsService-Q5GRB6IN.js	28.4 kB	+28.4 kB (new file)	🆕
./bundle/gemini-EL2LHDTN.js	552 kB	+552 kB (new file)	🆕
./bundle/interactiveCli-6SU6ED7K.js	1.66 MB	+1.66 MB (new file)	🆕
./bundle/oauth2-provider-MORXM7DN.js	9.16 kB	+9.16 kB (new file)	🆕

ℹ️ View Unchanged

Filename	Size	Change
./bundle/bundled/third_party/index.js	8 MB	0 B
./bundle/chunk-34MYV7JD.js	2.45 kB	0 B
./bundle/chunk-5AUYMPVF.js	858 B	0 B
./bundle/chunk-5PS3AYFU.js	1.18 kB	0 B
./bundle/chunk-664ZODQF.js	124 kB	0 B
./bundle/chunk-DAHVX5MI.js	206 kB	0 B
./bundle/chunk-IUUIT4SU.js	56.5 kB	0 B
./bundle/chunk-P52CHEF3.js	1.96 MB	0 B
./bundle/chunk-RJTRUG2J.js	39.8 kB	0 B
./bundle/cleanup-FHXBCAXY.js	0 B	-932 B (removed)	🏆
./bundle/devtools-36NN55EP.js	696 kB	0 B
./bundle/dist-T73EYRDX.js	356 B	0 B
./bundle/events-XB7DADIJ.js	418 B	0 B
./bundle/gemini.js	4.97 kB	0 B
./bundle/getMachineId-bsd-TXG52NKR.js	1.55 kB	0 B
./bundle/getMachineId-darwin-7OE4DDZ6.js	1.55 kB	0 B
./bundle/getMachineId-linux-SHIFKOOX.js	1.34 kB	0 B
./bundle/getMachineId-unsupported-5U5DOEYY.js	1.06 kB	0 B
./bundle/getMachineId-win-6KLLGOI4.js	1.72 kB	0 B
./bundle/memoryDiscovery-23MICZEL.js	980 B	0 B
./bundle/multipart-parser-KPBZEGQU.js	11.7 kB	0 B
./bundle/node_modules/@google/gemini-cli-devtools/dist/client/main.js	222 kB	0 B
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/_client-assets.js	229 kB	0 B
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/index.js	13.4 kB	0 B
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/types.js	132 B	0 B
./bundle/sandbox-macos-permissive-open.sb	890 B	0 B
./bundle/sandbox-macos-permissive-proxied.sb	1.31 kB	0 B
./bundle/sandbox-macos-restrictive-open.sb	3.36 kB	0 B
./bundle/sandbox-macos-restrictive-proxied.sb	3.56 kB	0 B
./bundle/sandbox-macos-strict-open.sb	4.82 kB	0 B
./bundle/sandbox-macos-strict-proxied.sb	5.02 kB	0 B
./bundle/src-QVCVGIUX.js	47 kB	0 B
./bundle/tree-sitter-7U6MW5PS.js	274 kB	0 B
./bundle/tree-sitter-bash-34ZGLXVX.js	1.84 MB	0 B
./bundle/cleanup-V7UUGI2L.js	932 B	+932 B (new file)	🆕

_{compressed-size-action}

[gemini-code-assist]

Code Review

This pull request introduces memory usage tests for large chat scenarios, including resuming large sessions. It updates the memory baseline schema to include externalBytes and optimizes FakeContentGenerator to read response files using streams to reduce memory overhead. Feedback focuses on improving the robustness of stream handling in tests by accounting for error events and using JSON.stringify instead of manual string concatenation for generating test data.

[sripasg]

From Gemini CLI review

Summary of Review
The PR introduces a streaming refactor for FakeContentGenerator to handle large response files, which is a positive change. However, the new memory tests have a fundamental architectural flaw: they measure the memory of the test runner
process, not the CLI process. Additionally, the mock data generator produces invalid JSON files, causing the "resume" tests to operate on empty sessions instead of the intended large history.

---

1. Critical: Fundamental Flaw in Memory Measurement
   The MemoryTestHarness uses process.memoryUsage() to take snapshots. Since TestRig.run() spawns the CLI as a separate process (child_process.spawn), process.memoryUsage() only tracks the memory of the Vitest test runner, not the Gemini CLI.

• Impact: The memory tests are not actually measuring the CLI's memory performance. A memory regression in the CLI would go undetected because the measurements are taken after the CLI process has already exited.
• Proof: The baseline for large-chat (which simulates ~1.6GB of data) is only 99MB in baselines.json. This is the typical overhead of the Vitest process, confirming the CLI's memory usage is ignored.

2. Critical: Invalid JSON Generation in Tests
   In memory-tests/memory-usage.test.ts, the generateSharedLargeChatData function contains a logic error when writing the session history file.

1 // memory-tests/memory-usage.test.ts
2 for (let i = 0; i < LARGE_CHAT_MSG_NUM; i++) {
3 // ... writes message objects ...
4 historyStream.write( ]\n}\n); // <--- BUG: This closes the JSON on EVERY iteration
5 }

• Impact: This produces an invalid JSON file. When the CLI tries to load this file during resume-large-chat, JSON.parse fails.
• Resulting Side Effect: ChatRecordingService.ts catches JSON parse errors and falls back to an empty conversation. Consequently, the "resume" tests are actually testing an empty session, completely bypassing the "large chat" scenario.

3. Correctness: FakeContentGenerator Streaming Refactor
   The move to createReadStream and readline in FakeContentGenerator.ts is correct and necessary to avoid Node.js's string length limits (~512MB to 1GB).

• Observation: While this fixes the loading crash, the class still stores all FakeResponse objects in a single array (private readonly responses: FakeResponse[]). For 1.6GB of responses, this array will consume ~1.6GB of heap memory. If the
  goal is true memory efficiency, the generator should read from the file on-demand rather than pre-loading everything.

---

Recommendations

1. Fix Memory Tracking: To properly test the CLI, the MemoryTestHarness needs to track the child process. Since tracking a child process's peak memory is difficult with spawn, a better approach might be to run the core logic in-process for
   memory tests (if architectural changes allow) or use a platform-specific tool to track the spawned PID's peak RSS.
2. Fix History Generation: Move the closing brackets ]\n}\n outside the for-loop and ensure commas are added between message objects.
3. Validate Baselines: Once tracking is fixed, the baselines should be regenerated. They should reflect the expected GB-scale memory usage for these scenarios.

[jacob314]

Approved. Now test cases look reasonable but we absolutely need to get a handle of why the memory baseline #s don't seem right.