feat(harness): show credentials for all in-use engines across agent configs

[aaight]

Summary

• Add listDistinctEnginesByProject repository query — queries agent_configs for all distinct non-null agent_engine values for a project, returning a string[] of engine IDs actively used by agent configs
• Add enginesInUse tRPC endpoint — a protectedProcedure that calls the new repository query, enabling the frontend and CLI to discover which engines need credentials configured
• Update Harness tab Engine Credentials card — fetches enginesInUse and merges with the project-level effectiveEngineId to show credential fields for ALL engines in use (not just the project default)
• Add credential status indicators to Agent Config list — adds a "⚠️ Missing credentials" badge on agent rows when the agent's engine override lacks configured credentials
• Update CLI projects show — displays which engines are actively used by agent configs in the project (beyond the project-level default)
• Unit tests — new tests for listDistinctEnginesByProject (repository) and enginesInUse (tRPC endpoint), plus fixes to existing CLI tests for the new endpoint dependency

Test plan

• Unit tests pass for listDistinctEnginesByProject (returns distinct engine IDs, excludes nulls, empty results)
• Unit tests pass for enginesInUse tRPC endpoint (org access, auth, empty response)
• All existing tests pass (7172 tests, 366 files)
• Lint passes (no errors)
• TypeScript type check passes (no errors)
• Manual testing:
  • Create project with claude-code as default engine
  • Add agent config with codex engine override
  • Navigate to Harness tab → both Claude Code and Codex credentials should appear
  • CLI cascade projects show <id> should list "codex" under "Agent Engines In Use"
  • Remove agent config engine override → Codex credentials disappear from Harness tab

Key decisions

• No DB migration needed — this is purely additive, using existing agent_engine column in agent_configs
• Frontend allEnginesInUse merges project default + per-agent overrides — ensures all needed credentials are displayed even when the project default engine changes
• Missing credentials badge only on agent rows with explicit engine overrides — avoids false positives for agents inheriting the project-level engine (since project-level credentials are already visible in the same Harness tab)
• selectDistinct added to mock DB helper — backward-compatible addition to createMockDb

Resolves: https://trello.com/c/vlJYL9eD/566-lets-make-all-engines-available-in-engine-configuration-in-dashboard-cli-on-per-project-basis-we-can-select-default-engine-and-c

🤖 Generated with Claude Code

🕵️ claude-code · claude-sonnet-4-6 · run details

[codecov]

Codecov Report

❌ Patch coverage is 96.15385% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/cli/dashboard/projects/show.ts	90.90%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

[nhopeatall]

Summary

Clean, well-structured feature with proper auth checks, good test coverage, and all CI passing. One should-fix issue around duplicated credential mappings that will drift out of sync.

Architecture & Design

• [SHOULD_FIX] Duplicated engine-to-credential mapping: ENGINE_CREDENTIAL_KEYS in project-agent-configs.tsx (line 487) duplicates the credential-to-engine mapping already defined by ENGINE_SECRETS in project-harness-form.tsx, but with different data. Specifically, ENGINE_CREDENTIAL_KEYS maps llmist to ['OPENROUTER_API_KEY'], but ENGINE_SECRETS only maps OPENROUTER_API_KEY to engines ['opencode'] — it does not include llmist. This creates a concrete UX inconsistency introduced by this PR:

  1. When an agent has a llmist engine override, the new agent row badge shows "Missing credentials" (because ENGINE_CREDENTIAL_KEYS says llmist needs OPENROUTER_API_KEY)
  2. But navigating to the Harness tab will NOT show the OPENROUTER_API_KEY field for llmist (because ENGINE_SECRETS does not include llmist in the OPENROUTER_API_KEY engines list)
  3. The user sees a warning they cannot act on from the same UI

  Fix options:

  • Extract a shared ENGINE_CREDENTIAL_MAP constant that both files import
  • Derive ENGINE_CREDENTIAL_KEYS from ENGINE_SECRETS programmatically
  • At minimum, add llmist to the OPENROUTER_API_KEY entry's engines array in ENGINE_SECRETS

Code Issues

Should Fix

• web/src/components/projects/project-agent-configs.tsx:487-492 — The duplicated mapping will become a maintenance burden. Every time an engine or credential is added/changed in ENGINE_SECRETS, this map needs manual updating in a separate file. The llmist discrepancy is already causing a real issue as described above.

Everything else looks good — the backend query is clean and correct (using selectDistinct with isNotNull filter), auth checks follow existing patterns, tests are thorough, CLI gracefully degrades with .catch(() => []), and the frontend properly merges project-level + agent-level engines for the credentials display.

🕵️ claude-code · claude-opus-4-6 · run details

[nhopeatall]

Summary

LGTM — Clean, well-structured feature addition with proper layering across backend, frontend, and CLI.

What I verified:

• Repository layer: listDistinctEnginesByProject correctly uses selectDistinct with isNotNull filter and the as string cast is safe given the isNotNull WHERE clause.
• tRPC endpoint: Properly guarded by protectedProcedure with verifyProjectOrgAccess, re-exported cleanly through settingsRepository.ts wildcard export.
• Shared module extraction: ENGINE_SECRETS + engineCredentialKeys computed once at module load — both project-harness-form.tsx and project-agent-configs.tsx share a single source of truth. Minor intentional behavioral change: OPENROUTER_API_KEY now also maps to llmist engine (correct, since llmist supports OpenRouter).
• Harness form credentials: allEnginesInUse merge logic deduplicates via Set, condition changed from !effectiveEngineId to allEnginesInUse.length === 0 which correctly handles the case where no project engine is selected but agents have overrides.
• Missing credentials badge: engineHasCredentials uses .some() semantics — correct since credential keys for an engine are alternatives (API key auth vs subscription auth), not all required simultaneously. Only shown on agent rows with explicit engine overrides to avoid false positives.
• CLI: enginesInUse query is gracefully wrapped with .catch(() => []) for backward compatibility with older backends. null value correctly renders as '—' via the printDetail formatter.
• Tests: Comprehensive coverage for repository, tRPC endpoint (happy path, empty, auth, access), and CLI mock wiring.
• CI: All 7 checks passing.

🕵️ claude-code · claude-opus-4-6 · run details