test(core): improve sandbox integration test coverage and fix OS-specific failures#25307
test(core): improve sandbox integration test coverage and fix OS-specific failures#25307
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the robustness of the Linux sandbox environment by introducing comprehensive integration tests and fixing a critical bug. The bug previously prevented write access to policy-authorized paths if they did not yet exist and the command was not explicitly recognized as a write operation, leading to read-only file system errors. The changes ensure that the sandbox consistently grants appropriate write permissions for allowed paths, aligning with the defined execution policy and improving overall sandbox reliability. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
Size Change: +617 B (0%) Total Size: 34.1 MB
ℹ️ View Unchanged
|
There was a problem hiding this comment.
Code Review
This pull request simplifies the Linux sandbox argument building by removing the isWriteCommand flag and consistently using --bind-try for parent directories of non-existent paths. Additionally, it significantly enhances the integration test suite for the SandboxManager, introducing comprehensive coverage for virtual commands, environment sanitization, sandbox mode transitions, recursive forbidden path protection, git worktree metadata security, and network access policies. I have no feedback to provide as no issues were identified in the changes.
galz10
left a comment
There was a problem hiding this comment.
Code Review
Scope: Pull Request #25307
This pull request significantly improves the sandbox integration tests by introducing better isolation (fresh workspace per test) and increasing coverage for various sandbox features (virtual commands, environment sanitization, etc.). It also addresses platform-specific bugs, notably by ensuring "governance files" are initialized on macOS to allow the Seatbelt profile to target them effectively, and by simplifying the Linux Bubblewrap argument builder.
Metadata Review
- Title/Description: The title
test(core): improve sandbox integration test coverage and fix OS-specific failuresis accurate but slightly undersells the impact, as it also includes core logic changes in theSandboxManagerimplementations for parity and reliability. The description is clear and explains the motivation for the changes well.
Concerns (Action Required)
-
[MacOsSandboxManager.ts]: Workspace Pollution: The PR adds logic to automatically create
.git,.gitignore, and.geminiignorein the workspace on every command preparation if they don't already exist. While this ensures Seatbelt can protect these paths, it results in hidden files/folders being created in any directory where the CLI is run (e.g., a user's home directory or downloads folder).- Suggestion: Consider creating these files only if they are missing and the security policy specifically requires their presence, or document this behavior clearly. Alternatively, if Seatbelt can block creation using
(deny file-write-create ...)on the literal path even if it doesn't exist, that would be preferable to "touching" the files.
- Suggestion: Consider creating these files only if they are missing and the security policy specifically requires their presence, or document this behavior clearly. Alternatively, if Seatbelt can block creation using
-
[bwrapArgsBuilder.ts]: Linux Security Regression: By removing the
isWriteCommandcheck and always using--bind-tryfor the parent directory of non-existent allowed paths, the sandbox now grants read-write access to the parent directory even for supposedly read-only commands.- Suggestion: If a command is explicitly intended to be read-only (e.g., a
__readvirtual command), we should use--ro-bind-tryfor the parent directory instead of--bind-try. This preserves the principle of least privilege.
- Suggestion: If a command is explicitly intended to be read-only (e.g., a
-
[MacOsSandboxManager.ts]: Initialization Inefficiency: The governance file initialization loop (calling
touch) runs insideprepareCommand. For long-running sessions with many tool calls, this results in redundantlstatcalls and filesystem checks for every single command.- Suggestion: Move the initialization to the
SandboxManagerconstructor or a one-timeinitializemethod, or cache the fact that initialization has already been performed for the current workspace.
- Suggestion: Move the initialization to the
-
[sandboxManager.integration.test.ts]: Temporary Directory Cleanup: The refactor to
beforeEachfor workspace creation is excellent for isolation. However, since cleanup happens inafterAll, all temporary workspaces for every test in the file will persist on disk until the entire suite finishes.- Suggestion: Move the
rmSynclogic for the currentworkspaceinto anafterEachblock to keep the disk footprint minimal during test execution, while keepingafterAllas a fail-safe for other temporary directories created viacreateTempDir.
- Suggestion: Move the
Nits (Suggestions)
- [MacOsSandboxManager.ts]:
touchImplementation: The checkif (fs.lstatSync(filePath)) return;is slightly redundant becauselstatSyncwill either throw or return a truthyStatsobject. A simplertry { fs.lstatSync(filePath); return; } catch {}is sufficient and avoids the unnecessary truthiness check. - [MacOsSandboxManager.ts]:
touchError Handling: Thecatchblock intouchignores all errors. It would be safer to specifically catchENOENTand let other errors (likeEACCES) bubble up to help diagnose permission issues in the test environment. - [bwrapArgsBuilder.ts]: Comment Accuracy: The comment "Bind the parent directory as read-write" is correct given the implementation change, but it highlights the security concern mentioned above.
|
DavidAPierce
left a comment
There was a problem hiding this comment.
LGTM. Thanks for tackling this!
Comments addressed, David gave approval
* refactor(plan): simplify policy priorities and consolidate read-only rules (google-gemini#24849) * feat(test-utils): add memory usage integration test harness (google-gemini#24876) * feat(memory): add /memory inbox command for reviewing extracted skills (google-gemini#24544) * chore(release): bump version to 0.39.0-nightly.20260408.e77b22e63 (google-gemini#24939) * fix(core): ensure robust sandbox cleanup in all process execution paths (google-gemini#24763) Co-authored-by: Spencer <spencertang@google.com> * chore: update ink version to 6.6.8 (google-gemini#24934) * Changelog for v0.38.0-preview.0 (google-gemini#24938) Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com> Co-authored-by: g-samroberts <samroberts@google.com> * chore: ignore conductor directory (google-gemini#22128) Co-authored-by: Coco Sheng <cocosheng@google.com> * Changelog for v0.37.0 (google-gemini#24940) Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com> Co-authored-by: Sam Roberts <158088236+g-samroberts@users.noreply.github.com> * feat(plan): require user confirmation for activate_skill in Plan Mode (google-gemini#24946) * feat(test-utils): add CPU performance integration test harness (google-gemini#24951) * fix(core): resolve windows symlink bypass and stabilize sandbox integration tests (google-gemini#24834) * test(sdk): add unit tests for GeminiCliSession (google-gemini#21897) * fix(cli): restore file path display in edit and write tool confirmations (google-gemini#24974) * fix(cli-ui): enable Ctrl+Backspace for word deletion in Windows Terminal (google-gemini#21447) * fix(core): dynamic session ID injection to resolve resume bugs (google-gemini#24972) * Update ink version to 6.6.9 (google-gemini#24980) * feat(core): refine shell tool description display logic (google-gemini#24903) * Generalize evals infra to support more types of evals, organization and queuing of named suites (google-gemini#24941) * fix(cli): optimize startup with lightweight parent process (google-gemini#24667) * refactor(sandbox): use centralized sandbox paths in macOS Seatbelt implementation (google-gemini#24984) * feat(cli): refine tool output formatting for compact mode (google-gemini#24677) * fix(sdk): skip broken sendStream tests to unblock nightly (google-gemini#25000) * refactor(core): use centralized path resolution for Linux sandbox (google-gemini#24985) * Support ctrl+shift+g (google-gemini#25035) * feat(core): refactor subagent tool to unified invoke_subagent tool (google-gemini#24489) * fix(core): add explicit git identity env vars to prevent sandbox checkpointing error (google-gemini#19775) Co-authored-by: David Pierce <davidapierce@google.com> * fix: respect hideContextPercentage when FooterConfigDialog is closed without changes (google-gemini#24773) Co-authored-by: Coco Sheng <cocosheng@google.com> * fix(cli): suppress unhandled AbortError logs during request cancellation (google-gemini#22621) * Automated documentation audit (google-gemini#24567) * feat(cli): implement useAgentStream hook (google-gemini#24292) Co-authored-by: Adam Weidman <adamfweidman@gmail.com> Co-authored-by: Adam Weidman <adamfweidman@google.com> * refactor(core): remove legacy subagent wrapping tools (google-gemini#25053) * refactor(plan) Clean default plan toml (google-gemini#25037) * fix(core): honor retryDelay in RetryInfo for 503 errors (google-gemini#25057) * fix(core): remediate subagent memory leaks using AbortSignal in MessageBus (google-gemini#25048) * feat(cli): wire up useAgentStream in AppContainer (google-gemini#24297) Co-authored-by: Adam Weidman <adamfweidman@gmail.com> Co-authored-by: Adam Weidman <adamfweidman@google.com> * feat(core): migrate chat recording to JSONL streaming (google-gemini#23749) * fix(core): clear 5-minute timeouts in oauth flow to prevent memory leaks (google-gemini#24968) * fix(sandbox): centralize async git worktree resolution and enforce read-only security (google-gemini#25040) * feat(test): add high-volume shell test and refine perf harness (google-gemini#24983) * fix(core): silently handle EPERM when listing dir structure (google-gemini#25066) * Changelog for v0.37.1 (google-gemini#25055) Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com> * fix: decode Uint8Array and multi-byte UTF-8 in API error messages (google-gemini#23341) Co-authored-by: Coco Sheng <cocosheng@google.com> * Automated documentation audit results (google-gemini#22755) * debugging(ui): add optional debugRainbow setting (google-gemini#25088) * fix: resolve lifecycle memory leaks by cleaning up listeners and root closures (google-gemini#25049) * docs(cli): updates f12 description to be more precise (google-gemini#15816) * fix(cli): mark /settings as unsafe to run concurrently (google-gemini#25061) * fix(core): remove buffer slice to prevent OOM on large output streams (google-gemini#25094) * feat(core): persist subagent agentId in tool call records (google-gemini#25092) * chore(core): increase codebase investigator turn limits to 50 (google-gemini#25125) * refactor(core): consolidate execute() arguments into ExecuteOptions (google-gemini#25101) * feat(core): add Strategic Re-evaluation guidance to system prompt (google-gemini#25062) * fix(core): preserve shell execution config fields on update (google-gemini#25113) * docs: add vi shortcuts and clarify MCP sandbox setup (google-gemini#21679) Co-authored-by: Jenna Inouye <jinouye@google.com> * fix(cli): pass session id to interactive shell executions (google-gemini#25114) * fix(cli): resolve text sanitization data loss due to C1 control characters (google-gemini#22624) * feat(core): add large memory regression test (google-gemini#25059) * fix(core): resolve PTY exhaustion and orphan MCP subprocess leaks (google-gemini#25079) * chore: switch from keytar to @github/keytar (google-gemini#25143) * chore(deps): update vulnerable dependencies via npm audit fix (google-gemini#25140) * perf(sandbox): optimize Windows sandbox initialization via native ACL application (google-gemini#25077) * fix: improve audio MIME normalization and validation in file reads (google-gemini#21636) Co-authored-by: Coco Sheng <cocosheng@google.com> * docs: Update docs-audit to include changes in PR body (google-gemini#25153) * docs: correct documentation for enforced authentication type (google-gemini#25142) * fix(cli): exclude update_topic from confirmation queue count (google-gemini#24945) * Memory fix for trace's streamWrapper. (google-gemini#25089) * fix(core): fix quota footer for non-auto models and improve display (google-gemini#25121) * docs(contributing): clarify self-assignment policy for issues (google-gemini#23087) * feat(core): add skill patching support with /memory inbox integration (google-gemini#25148) * Stop suppressing thoughts and text in model response (google-gemini#25073) * fix(release): prefix git hash in nightly versions to prevent semver normalization (google-gemini#25304) * feat(cli): extract QuotaContext and resolve infinite render loop (google-gemini#24959) * refactor(core): extract and centralize sandbox path utilities (google-gemini#25305) Co-authored-by: David Pierce <davidapierce@google.com> * feat(ui): added enhancements to scroll momentum (google-gemini#24447) * fix(core): replace custom binary detection with isbinaryfile to correctly handle UTF-8 (U+FFFD) (google-gemini#25297) * feat(agent): implement tool-controlled display protocol (Steps 2-3) (google-gemini#25134) * Stop showing scrollbar unless we are in terminalBuffer mode (google-gemini#25320) * fix(core): expose GEMINI_PLANS_DIR to hook environment (google-gemini#25296) Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * feat: support auth block in MCP servers config in agents (google-gemini#24770) * feat(core): implement silent fallback for Plan Mode model routing (google-gemini#25317) * fix: correct redirect count increment in fetchJson (google-gemini#24896) Co-authored-by: Tommaso Sciortino <sciortino@gmail.com> * fix(core): prevent secondary crash in ModelRouterService finally block (google-gemini#25333) * feat(core): introduce decoupled ContextManager and Sidecar architecture (google-gemini#24752) * docs(core): update generalist agent documentation (google-gemini#25325) * chore(mcp): check MCP error code over brittle string match (google-gemini#25381) * test(core): improve sandbox integration test coverage and fix OS-specific failures (google-gemini#25307) Co-authored-by: David Pierce <davidapierce@google.com> * feat(plan): update plan mode prompt to allow showing plan content (google-gemini#25058) * fix(core): use debug level for keychain fallback logging (google-gemini#25398) * feat(test): add a performance test in asian language (google-gemini#25392) * feat(cli): enable mouse clicking for cursor positioning in AskUser multi-line answers (google-gemini#24630) * fix(core): detect kmscon terminal as supporting true color (google-gemini#25282) Co-authored-by: Adib234 <30782825+Adib234@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * ci: add agent session drift check workflow (google-gemini#25389) * use macos-latest-large runner where applicable. (google-gemini#25413) * Changelog for v0.37.2 (google-gemini#25336) Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com> * chore(release): v0.39.0-preview.0 * fix(patch): cherry-pick a4e98c0 to release/v0.39.0-preview.0-pr-25138 to patch version v0.39.0-preview.0 and create version 0.39.0-preview.1 (google-gemini#25766) Co-authored-by: Mahima Shanware <mahima.shanware@gmail.com> * chore(release): v0.39.0-preview.1 * fix(patch): cherry-pick d6f88f8 to release/v0.39.0-preview.1-pr-25670 to patch version v0.39.0-preview.1 and create version 0.39.0-preview.2 (google-gemini#25776) Co-authored-by: Adam Weidman <65992621+adamfweidman@users.noreply.github.com> * chore(release): v0.39.0-preview.2 * chore(release): v0.39.0 --------- Co-authored-by: ruomeng <ruomeng@google.com> Co-authored-by: Sri Pasumarthi <111310667+sripasg@users.noreply.github.com> Co-authored-by: Sandy Tao <sandytao520@icloud.com> Co-authored-by: gemini-cli-robot <gemini-cli-robot@google.com> Co-authored-by: Emily Hedlund <ehedlund@google.com> Co-authored-by: Spencer <spencertang@google.com> Co-authored-by: Jacob Richman <jacob314@gmail.com> Co-authored-by: gemini-cli-robot <224641728+gemini-cli-robot@users.noreply.github.com> Co-authored-by: g-samroberts <samroberts@google.com> Co-authored-by: JAYADITYA <96861162+JayadityaGit@users.noreply.github.com> Co-authored-by: Coco Sheng <cocosheng@google.com> Co-authored-by: Sam Roberts <158088236+g-samroberts@users.noreply.github.com> Co-authored-by: Adamya Singh <adamyasingh54@gmail.com> Co-authored-by: Jarrod Whelan <150866123+jwhelangoog@users.noreply.github.com> Co-authored-by: dogukanozen <dogukannozen@hotmail.com> Co-authored-by: Tommaso Sciortino <sciortino@gmail.com> Co-authored-by: Christian Gunderman <gundermanc@google.com> Co-authored-by: Sehoon Shon <sshon@google.com> Co-authored-by: Abhi <43648792+abhipatel12@users.noreply.github.com> Co-authored-by: MD. MOHIBUR RAHMAN <35300157+mrpmohiburrahman@users.noreply.github.com> Co-authored-by: David Pierce <davidapierce@google.com> Co-authored-by: chernistry <73943355+chernistry@users.noreply.github.com> Co-authored-by: euxaristia <25621994+euxaristia@users.noreply.github.com> Co-authored-by: Michael Bleigh <mbleigh@mbleigh.com> Co-authored-by: Adam Weidman <adamfweidman@gmail.com> Co-authored-by: Adam Weidman <adamfweidman@google.com> Co-authored-by: Yuna Seol <yunaseol@gmail.com> Co-authored-by: June <kimjune01@gmail.com> Co-authored-by: Aishanee Shah <aishaneeshah@google.com> Co-authored-by: Jason Matthew Suhari <jasonmatthewsuhari@gmail.com> Co-authored-by: Christopher Thomas <cobekgn@gmail.com> Co-authored-by: Jenna Inouye <jinouye@google.com> Co-authored-by: cynthialong0-0 <82900738+cynthialong0-0@users.noreply.github.com> Co-authored-by: M Junaid Shaukat <154750865+junaiddshaukat@users.noreply.github.com> Co-authored-by: Abhijit Balaji <abhijitbalaji@google.com> Co-authored-by: Mark Griffith <anthraxmilkshake@hotmail.com> Co-authored-by: Jack Wotherspoon <jackwoth@google.com> Co-authored-by: Jesse Rosenstock <jesse.rosenstock@gmail.com> Co-authored-by: Adib234 <30782825+Adib234@users.noreply.github.com> Co-authored-by: Dev Randalpura <devrandalpura@google.com> Co-authored-by: Anjaligarhwal <anjaligarhwal1610@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Tanmay Vartak <9002434+TanmayVartak@users.noreply.github.com> Co-authored-by: Jerop Kipruto <jerop@google.com> Co-authored-by: Kevin Zhao <kevin8093@126.com> Co-authored-by: joshualitt <joshualitt@google.com> Co-authored-by: Clay <claygeo6@gmail.com> Co-authored-by: Adam Weidman <65992621+adamfweidman@users.noreply.github.com> Co-authored-by: Mahima Shanware <mahima.shanware@gmail.com> Co-authored-by: codex-bot <spigelly+gh-bot@gmail.com>
Summary
This PR significantly improves the integration test coverage for the sandbox environments across all major operating systems (Linux, macOS, Windows). In the process of expanding this test suite to rigorously verify sandbox boundaries and policies, several OS-specific failures were identified and resolved, including initialization inefficiencies and a Bubblewrap permission bug on Linux.
Details
Test Coverage Enhancements (Primary)
sandboxManager.integration.test.tsto comprehensively exercise sandbox boundaries across Linux, macOS, and Windows.beforeAlltobeforeEachfor workspace initialization. Each test case now receives a fresh, isolated temporary directory to prevent state leakage (critical for Windows ACL checks). Added anafterEachcleanup block that specifically tracks and removes isolated test directories, drastically reducing the disk footprint during testing.bwrapArgsBuilder(Linux) andMacOsSandboxManager(macOS).Sandbox Logic Fixes (Secondary)
bwrap) Security:--bind-try) to any path inpolicyAllowedand its parent directory, unless the command is an explicit read-only virtual command (__read), in which case the parent directory is safely bound as read-only.bwrap) & Windows Optimization:ensureGovernanceFilesExist) to ensure that governance files (e.g.,.git,.gitignore) are secured and verified exactly once per session workspace, rather than redundantly on every single command execution. This reduces disk I/O and improves performance for long-running sessions.policy.allowedPathsinstead of non-existent target files, satisfying the Windows Sandbox requirement that granular access can only be granted to existing filesystem objects.ensureHelperCompiled) to be a globally static initialization, ensuring it only compiles once per Node process.Related Issues
Fixes #24932.
How to Validate
Windows, Linux, macOS
Run integration tests:
npm run test -w @google/gemini-cli-core -- src/services/sandboxManager.integration.test.tsLinux
Verify unit tests:
npm run test -w @google/gemini-cli-core -- src/sandbox/linux/bwrapArgsBuilder.test.ts