test(backends): add engine contract test for all registered engines#837
test(backends): add engine contract test for all registered engines#837
Conversation
nhopeatall
left a comment
There was a problem hiding this comment.
Summary
Clean, well-structured contract test that validates all 4 built-in engines against the AgentEngine interface. One duplicated test doesn't verify what its name claims.
Should Fix
tests/unit/backends/engine-contract.test.ts, lines 25-30 — The test "registers exactly the expected engines" is functionally identical to "registers all 4 built-in engines" (lines 17-22). Both iterate EXPECTED_ENGINE_IDS and assert each is contained in registeredIds:
// Test 1 (line 17): "registers all 4 built-in engines"
const registeredIds = getRegisteredEngines();
for (const id of EXPECTED_ENGINE_IDS) {
expect(registeredIds).toContain(id);
}
// Test 2 (line 25): "registers exactly the expected engines"
const registeredIds = getRegisteredEngines();
for (const id of EXPECTED_ENGINE_IDS) {
expect(registeredIds).toContain(id);
}The name "exactly" implies it should also verify no unexpected engines are registered, but it doesn't. To make this test meaningful, add an assertion like:
expect(registeredIds).toHaveLength(EXPECTED_ENGINE_IDS.length);Or remove the duplicate entirely.
🕵️ claude-code · claude-opus-4-6 · run details
…gful Add toHaveLength assertion so the test verifies no unexpected engines are registered, not just that the expected ones are present. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
nhopeatall
left a comment
There was a problem hiding this comment.
LGTM — Clean contract test that validates all 4 built-in engines against the AgentEngine interface. Test isolation is sound (vitest uses pool: 'forks', so the module-level registry state won't leak across test files). The describe.each pattern keeps per-engine validation DRY, optional member checks correctly guard with \!== undefined, and the hardcoded EXPECTED_ENGINE_IDS array will catch newly added engines via the length assertion. CI green.
🕵️ claude-code · claude-opus-4-6 · run details
Summary
tests/unit/backends/engine-contract.test.ts— a contract test that validates all 4 built-in engines against theAgentEngineinterfacedescribe.eachfor consistent per-engine validationgetEngineCatalog()returns definitions for all registered enginesWhat is tested
registerBuiltInEngines()successfully registers all 4 engines:llmist,claude-code,codex,opencodedefinitionhas the required fields:id,label,description,capabilities(array),modelSelection,logLabeldefinition.idmatches its registry keyexecuteis a function on each enginesupportsAgentType()is a function that returns a boolean for known agent types (implementation,review,splitting)resolveModelis a function if present (validates Story chore: update llmist and zangief dependencies #3 compliance)beforeExecute/afterExecuteare functions if present (validates Story feat: add UpdateChecklistItem gadget and improve agent workflow #5 compliance)getEngineCatalog()returns definitions for all registered engines and references the same objects as the registryTest plan
tests/unit/backends/engine-contract.test.tsexists and passes (49 tests)npm run typecheckpassesnpm run lintpassesCard: https://trello.com/c/69b57342a440fa1ff9a93706
🕵️ claude-code · claude-sonnet-4-6 · run details