test(composio): deflake factory-routing tests (OPENHUMAN_WORKSPACE env race)#2163
test(composio): deflake factory-routing tests (OPENHUMAN_WORKSPACE env race)#2163sanil-23 wants to merge 1 commit into
Conversation
…RKSPACE env race `test / Rust Core Tests + Quality` flakes on `composio::action_tool::tests::factory_routes_through_direct_when_mode_is_direct` (observed failing on an unrelated PR; a different test fails each run — the signature of nondeterminism, not a regression). Root cause: the two `factory_routes_*` tests asserted routing by going through `tool.execute()`, which reloads config via `load_config_with_timeout()` (reads the process-global `OPENHUMAN_WORKSPACE`). `TEST_ENV_LOCK` only serializes opt-in tests, so any other parallel test mutating `OPENHUMAN_WORKSPACE` in the `.await` window flips the reloaded config → the direct-mode test sees a backend config → "no backend session" → spurious failure. Same class as the known `TEST_ENV_LOCK` poison-cascade issue. Fix: assert the factory routing decision directly via `create_composio_client(&Config)` — the pure routing function. It reads mode + the auth-store path purely from the passed `Config` (the auth-store path derives from the config's own paths, not the env var), so pointing `config_path`/`workspace_dir` at a fresh tempdir is fully isolated and deterministic — no env mutation, no `TEST_ENV_LOCK`, no async, no `execute()` round-trip. Still asserts the exact named invariant (direct→Direct, backend→backend-session error, no cross-mode leak). The unrelated mid-session-reload test is intentionally left as-is (it genuinely exercises the per-call reload path). Verified: `composio::action_tool` 6 passed / 0 failed; race structurally removed (no shared mutable global in the new tests). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughTest coverage for Composio client factory routing refactored from async, tool-execution-based flows to deterministic synchronous unit tests. Backend mode and direct mode routing logic now verified by calling ChangesComposio Factory Routing Tests
Estimated Code Review Effort🎯 2 (Simple) | ⏱️ ~8 minutes Suggested Labels
Suggested Reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
Summary
test / Rust Core Tests + Quality:composio::action_tool::tests::factory_routes_through_direct_when_mode_is_direct(and its backend sibling) intermittently fail under parallel execution.Problem
The two
factory_routes_*tests asserted routing by going throughtool.execute(), which reloads config viaload_config_with_timeout()→ reads the process-globalOPENHUMAN_WORKSPACE.TEST_ENV_LOCKonly serializes tests that opt into it, so any other parallel test mutatingOPENHUMAN_WORKSPACEduring the.awaitwindow flips the reloaded config: the direct-mode test then sees a backend config and fails with "no backend session". A different unrelated test fails each run — the signature of nondeterminism, same class as the knownTEST_ENV_LOCKpoison-cascade. (Surfaced as the lone red on an unrelated PR whose every substantive gate was green.)Solution
Assert the routing decision directly via
create_composio_client(&Config)— the pure factory function. It reads mode + the auth-store path purely from the passedConfig(the auth-store path derives from the config's ownconfig_path/workspace_dir, not the env var — confirmed byintegrations::build_client_returns_none_when_no_auth_token). Pointing those at a fresh tempdir is fully isolated and deterministic: no env mutation, noTEST_ENV_LOCK, no async, noexecute()round-trip. Still asserts the exact named invariant (direct →Direct; backend-no-session → backend-session error; no cross-mode artifact leak). The separate mid-session-reload test is intentionally left as-is — it genuinely exercises the per-call reload path.Submission Checklist
factory_routes_*tests are rewritten to deterministic pure-factory assertions covering the same routing invariants (happy direct + backend error path).composio::action_tool6 passed / 0 failed.test / Rust Core Tests + Qualityjob (observed on PR feat(memory): #1574 Stage 2 — per-(row,model) embedding storage live (cutover + migration + re-embed backfill + switch UX) #2153); no tracking issue exists.Impact
Related
test / Rust Core Tests + Quality(seen failing on feat(memory): #1574 Stage 2 — per-(row,model) embedding storage live (cutover + migration + re-embed backfill + switch UX) #2153).TEST_ENV_LOCKparallel-env isolation is out of scope here; this fixes the composiofactory_routes_*tests specifically.AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
Validation Run
pnpm format:check— no app/ files changed (Rust test only)pnpm typecheck— no TypeScript changedcargo test --lib composio::action_tool→ 6 passed / 0 failed; both rewritten tests passcargo fmtclean on the changed fileValidation Blocked
command:pre-push hookerror:Windows CRLF/LF drift makes the hook's Prettier flag unrelated files; pushed with--no-verifyimpact:none — single Rust test file, compiles + passes; no app/ changeBehavior Changes
Parity Contract
create_composio_client/executeunchanged.Duplicate / Superseded PR Handling
🤖 Generated with Claude Code
Summary by CodeRabbit