Skip to content

Conversation

@ammar-agent
Copy link
Collaborator

@ammar-agent ammar-agent commented Feb 12, 2026

Summary

Eliminate the TEST_INTEGRATION env var and shouldRunIntegrationTests() per-file boilerplate from 72 files. Phase 3 of the integration test restructuring (Phase 1: #2370, Phase 2: #2389).

Background

Every integration test file repeated the same pattern:

import { shouldRunIntegrationTests } from "...";
const describeIntegration = shouldRunIntegrationTests() ? describe : describe.skip;

This conflated "integration test" with "needs real AI API keys" — two different concerns. Tests in tests/ are integration tests by definition; whether they need API keys should be expressed differently.

Implementation

Infrastructure (4 files)

File Change
tests/testUtils.ts Removed shouldRunIntegrationTests(). Simplified validateApiKeys()/getApiKey() to not depend on env var.
tests/ipc/setup.ts createTestEnvironment({ seedDummyKeys?: boolean }) — explicit parameter replaces env var check. Default true seeds dummy keys for UI tests; setupProviders() overwrites for real-key tests.
tests/setup.ts Removed conditional preload block. Tests needing AI SDK preloading call preloadTestModules() in their own beforeAll.
jest.config.js Fixed 120s default timeout (replaces TEST_INTEGRATION ternary). AI tests set jest.setTimeout(600_000) individually.

Test files (65 files)

  • 22 UI tests: Removed shouldRunIntegrationTests import + describeIntegration wrapper → bare describe()
  • 38 IPC tests: Same removal. 26 AI-calling tests got jest.setTimeout(600_000).
  • Compound-condition tests: anthropicCacheStrategy, ollama, system1BashCompaction now check only key/service availability (not TEST_INTEGRATION)
  • src/node/services/streamManager.test.ts: Replaced env var check with key-availability gate (describeWithApi)
  • tests/run/smoke.integration.test.ts: Same pattern removal + jest.setTimeout
  • Removed stale committed tests/testUtils.js + .js.map artifacts
  • Cleaned up stale // Skip all tests if TEST_INTEGRATION is not set comments and Docker skip messages

Developer documentation & CI (3 files)

File Change
.mux/skills/tests/SKILL.md Updated test layout docs to reflect subfolder structure from Phases 1-2. Removed TEST_INTEGRATION=1 / shouldRunIntegrationTests() guidance. Added guidance on jest.setTimeout(600_000) and validateApiKeys() for AI tests. Updated example test paths.
.github/workflows/pr.yml Removed all TEST_INTEGRATION=1 prefixes from CI jest invocations (4 occurrences) and TEST_INTEGRATION env var from Windows smoke test job.
Makefile Removed TEST_INTEGRATION=1 from test-integration target.

Net impact: -379 lines across 72 files

Validation

  • make typecheck clean
  • make lint clean
  • make fmt-check clean
  • ✅ 35/35 UI test suites pass without TEST_INTEGRATION=1
  • ✅ Non-AI IPC tests pass (workspace/create, workspace/remove, doubleRegister, windowTitle)
  • AI IPC tests require real API keys — validateApiKeys() fails fast when keys missing

Risks

  • jest.config.js timeout raised from 10s → 120s for all tests. This is safe because unit tests in src/ run under bun test (not jest), and integration tests in tests/ universally need >10s for service container setup.
  • tests/runtime/ files retain their own local shouldRunIntegrationTests() and TEST_INTEGRATION check — these are self-contained, Docker-gated tests with separate infrastructure. Out of scope for this change.
  • src/ files that check TEST_INTEGRATION for telemetry suppression or network-calling unit test gating are unrelated concerns and left untouched.

Generated with mux • Model: anthropic:claude-opus-4-6 • Thinking: xhigh • Cost: $25.47

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e6d66bbfdd

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +14 to +15
const hasAnthropicKey = Boolean(process.env.ANTHROPIC_API_KEY);
const describeWithApi = hasAnthropicKey ? describe : describe.skip;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require explicit opt-in for real-API StreamManager test

This change gates the real Anthropic test on ANTHROPIC_API_KEY alone, so it now runs during bun test src whenever that env var is present. Because make test/make test-unit executes bun test src (Makefile lines 340-343), developers or CI environments with a key exported will unexpectedly hit live API calls in what is supposed to be a unit-test path, causing slow/flaky/costly runs and potential failures from transient API issues.

Useful? React with 👍 / 👎.

Remove shouldRunIntegrationTests() gating from 60+ test files across
tests/ipc/, tests/ui/, tests/run/, and src/. Phase 3 of integration
test restructuring.

Infrastructure changes:
- tests/testUtils.ts: Remove shouldRunIntegrationTests(). Simplify
  validateApiKeys() and getApiKey() to not depend on env var.
- tests/ipc/setup.ts: createTestEnvironment() now accepts
  { seedDummyKeys?: boolean } option instead of checking env var.
  Always seeds dummy keys by default (tests using real API keys call
  setupProviders() which overwrites them).
- tests/setup.ts: Remove conditional preload block. Tests needing
  AI SDK preloading call preloadTestModules() in their own beforeAll.
- jest.config.js: Replace TEST_INTEGRATION ternary with fixed 120s
  timeout. AI tests set jest.setTimeout(600_000) individually.

Test file changes:
- Remove shouldRunIntegrationTests import + describeIntegration
  pattern from all 22 UI test files and 38 IPC test files
- Replace conditional describe wrappers with bare describe()
- Add jest.setTimeout(600_000) to 26 AI-calling test files
- Files with compound conditions (anthropicCacheStrategy, ollama,
  system1BashCompaction) now check only key/service availability
- src/node/services/streamManager.test.ts: Replace env var check
  with key-availability gate for its API integration section
- Remove stale compiled testUtils.js/.js.map artifacts

Net: -323 lines of boilerplate removed across 69 files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant