Skip to content

Data-Driven Integration Dispatch: Implementation Plan #492

@danielmeppiel

Description

@danielmeppiel

Data-Driven Integration Dispatch: Implementation Plan

Problem Statement

PR #489 correctly identifies that install.py's ~200-line if-chain is brittle (missing guards caused --target opencode to write to .github/). However, the proposed INTEGRATION_DISPATCH registry fixes dispatch routing without addressing the root cause: the integrators themselves have 20+ near-identical per-target methods (70-95% boilerplate). The registry also introduces triple-encoding of path data (KNOWN_TARGETS + registry + method bodies).

The better fix: make integrators consume TargetProfile data directly, collapsing per-target methods into parameterized ones. KNOWN_TARGETS becomes the single source of truth. SkillIntegrator already proves this pattern works.

What To Do With PR #489

PR #489 was authored by Copilot (bot). Close it with a note that the bug (#470) is addressed by our own PR using a data-driven approach instead of a registry. Reference the new PR in the close comment. No contributor attribution needed.


Agent Team & Staffing

Team Composition

Role Agent Type Persona Responsibility
Tech Lead general-purpose (Opus 4.6) Python architect, owns design decisions, resolves cross-cutting concerns Leads Wave 1 design, reviews all waves, owns install.py + uninstall/engine.py changes
Integrator Specialist A general-purpose (Opus 4.6) File integrator expert, deep knowledge of BaseIntegrator + collision detection CommandIntegrator + AgentIntegrator refactor (Waves 2-3)
Integrator Specialist B general-purpose (Opus 4.6) Content transform + format_id expert InstructionIntegrator + HookIntegrator wrapper (Wave 3)
Test Engineer general-purpose (Opus 4.6) Test coverage, regression suites, exhaustiveness checks New tests for all waves, existing test migration
DX/UX Expert cli-logging-ux skill Developer experience specialist, CLI output quality, logging standards Audits ALL user-facing output in refactored code. Ensures every log, warning, error, progress message, and diagnostic is actionable, consistent with STATUS_SYMBOLS conventions, and world-class. Reviews Wave 1 output format as template for all subsequent waves.
Code Reviewer code-review High-signal review, catches bugs, security, logic errors Reviews each wave's output before merge

Leadership

Tech Lead is the decision-maker. They:

  • Define the integrate_*_for_target() method signature contract in Wave 1
  • Resolve design tensions (e.g., how HookIntegrator bridges to the new interface)
  • Own the CLI dispatch layer (install.py, uninstall/engine.py) — the riskiest cross-cutting change
  • Make final call on test sufficiency before shipping

DX/UX Expert has veto power on any user-facing output. They:

  • Audit every console message, warning, and error produced by refactored integration methods
  • Ensure all output follows STATUS_SYMBOLS conventions ([+], [!], [x], [i], [*], [>])
  • Verify logging is actionable (tells the user what happened AND what to do about it)
  • Check that verbose/quiet modes behave correctly in the new dispatch loop
  • Validate diagnostic summaries (DiagnosticCollector) surface meaningful info for debugging
  • Set the DX standard in Wave 1 that all subsequent waves must follow

Waves of Work

Wave 1: Foundation (Design Contract + CommandIntegrator)

Goal: Establish the integrate_*_for_target(target, ...) contract and prove it with the simplest integrator.

Tasks:

ID Task Owner Depends On
W1.1 Define integrate_for_target() and sync_for_target() method signatures on BaseIntegrator (or as convention) Tech Lead
W1.2 Refactor CommandIntegrator: collapse 2 integrate + 2 sync methods into parameterized versions Integrator Specialist A W1.1
W1.3 Write tests for CommandIntegrator new API (real + synthetic TargetProfile) Test Engineer W1.2
W1.4 Migrate existing 21 command_integrator tests to new API Test Engineer W1.2
W1.5 DX/UX audit of CommandIntegrator output: every log message, warning, error, progress indicator in integrate_commands_for_target() and sync_for_target(). Set the output quality template for all subsequent waves. DX/UX Expert W1.2

Panel Review: After W1.5, Code Reviewer reviews the CommandIntegrator diff. Tech Lead validates the method signature contract is clean and extensible. DX/UX Expert confirms all CLI output meets world-class standards and the output template is locked for subsequent waves.

Exit Criteria: CommandIntegrator passes all existing + new tests. Method signature is locked for other integrators. CLI output template is approved by DX/UX Expert.


Wave 2: Integrator Collapse (Agent + Instruction)

Goal: Apply the proven pattern to the two remaining file-copy integrators. Parallel work.

Tasks:

ID Task Owner Depends On
W2.1 Refactor AgentIntegrator: collapse 4 integrate + 4 sync + 3 filename methods. Split multi-target integrate_package_agents() into single-target-per-call Integrator Specialist A W1.4
W2.2 Refactor InstructionIntegrator: collapse 2 integrate + 2 sync methods. Wire format_id for cursor_rules transform dispatch Integrator Specialist B W1.4
W2.3 Write tests for AgentIntegrator new API (all 4 target profiles + synthetic + multi-target split verification) Test Engineer W2.1
W2.4 Write tests for InstructionIntegrator new API (format_id dispatch, cursor transform) Test Engineer W2.2
W2.5 Migrate existing 91 agent_integrator tests Test Engineer W2.1
W2.6 Migrate existing 35 instruction_integrator tests Test Engineer W2.2
W2.7 Add HookIntegrator thin wrapper: integrate_hooks_for_target(target, ...) that delegates to existing per-target methods by target.name Integrator Specialist B W1.5
W2.8 DX/UX audit of Agent + Instruction + Hook integrator output: verify all log messages, warnings, errors follow the Wave 1 template. Audit diagnostic messages for multi-target agent split (are users told which target failed?). Ensure format_id transform messages are actionable. DX/UX Expert W2.1, W2.2, W2.7

Parallelism: W2.1 and W2.2 run in parallel (independent integrators). W2.7 is independent of both. W2.8 runs after all three integrator refactors complete.

Panel Review: After W2.8, Code Reviewer reviews the full integrator diff. Tech Lead validates that the multi-target agent split is correct and no cross-target side effects exist. DX/UX Expert confirms output consistency across all integrators.

Exit Criteria: All 3 refactored integrators + hook wrapper pass all existing + new tests. format_id is consumed for the first time. All CLI output follows approved template.


Wave 3: Dispatch & Partition (install.py + uninstall + partition)

Goal: Replace the CLI dispatch layer and partition logic. This is the highest-risk wave — it touches the orchestration layer.

Tasks:

ID Task Owner Depends On
W3.1 Replace _integrate_package_primitives() if-chain with target×primitive dispatch loop. Add PRIMITIVE_TO_INTEGRATOR dict. Delete boolean flags. Tech Lead W2.5, W2.6, W2.7
W3.2 Update _sync_integrations_after_uninstall() to target-driven sync loop Tech Lead W3.1
W3.3 Replace partition_managed_files() hardcoded buckets with dynamic generation from KNOWN_TARGETS Tech Lead W3.2
W3.4 Delete should_integrate_vscode/claude/opencode from target_detection.py Tech Lead W3.1
W3.5 Write target-gating regression tests (opencode-only skips .github/, cursor-only skips others, etc.) Test Engineer W3.1
W3.6 Write exhaustiveness test: every (target, primitive) in KNOWN_TARGETS has a dispatch path Test Engineer W3.1
W3.7 Write partition parity test: dynamic partition matches old hardcoded buckets for all known prefixes Test Engineer W3.3
W3.8 DX/UX audit of dispatch loop output: audit the new target x primitive dispatch loop logging (which targets are active, which primitives are being integrated, what was skipped and why). Ensure --verbose shows per-target detail, default shows summary. Audit uninstall sync messages. Verify DiagnosticCollector captures target-level diagnostics for troubleshooting. DX/UX Expert W3.1, W3.2

Panel Review: After W3.8, Code Reviewer does full review of install.py + uninstall/engine.py. DX/UX Expert confirms dispatch loop logging is world-class. This is the critical gate.

Exit Criteria: Full unit suite passes (uv run pytest tests/unit tests/test_console.py -x). All target-gating regressions pass. Partition parity confirmed. Dispatch loop output approved by DX/UX Expert.


Wave 4: Cleanup & Ship

Goal: Final cleanup, delete dead code, full validation, ship.

Tasks:

ID Task Owner Depends On
W4.1 Delete all old per-target methods from integrators (now unreachable) Integrator Specialist A W3.4
W4.2 Clean up imports, update __init__.py exports Integrator Specialist B W4.1
W4.3 Update CHANGELOG.md with the changes Tech Lead W4.2
W4.4 Run full test suite, verify 534+ tests pass Test Engineer W4.2
W4.5 Final DX/UX audit — holistic pass across entire PR: end-to-end install/uninstall output consistency, error message quality, verbose vs quiet behavior, diagnostic summary completeness. Ensure the refactor didn't regress any user-facing output quality. DX/UX Expert W4.2
W4.6 Final Code Review — full PR diff Code Reviewer W4.4, W4.5
W4.7 Update docs if any user-facing behavior changed (check Starlight pages) Tech Lead W4.6

Exit Criteria: Full suite green. Code review approved. DX/UX audit passed. CHANGELOG updated. PR ready to merge.


Task Graph & Critical Path

Wave 1 (Foundation):
  W1.1 --> W1.2 --> W1.3 + W1.4 + W1.5 --> [Panel Review 1]

Wave 2 (Integrator Collapse):       +- W2.1 --> W2.3 + W2.5 -+
  [Panel Review 1] -->               +- W2.2 --> W2.4 + W2.6 -+--> W2.8 --> [Panel Review 2]
                                     +- W2.7 -----------------+

Wave 3 (Dispatch):
  [Panel Review 2] --> W3.1 --> W3.2 --> W3.3 --> W3.7
                         |        |                 +--> W3.8 --> [Panel Review 3]
                         +--> W3.4
                         +--> W3.5
                         +--> W3.6

Wave 4 (Cleanup):
  [Panel Review 3] --> W4.1 --> W4.2 --> W4.3 + W4.4 + W4.5 --> W4.6 --> W4.7

Critical Path: W1.1 -> W1.2 -> W1.5 -> W2.1 -> W2.8 -> W3.1 -> W3.3 -> W3.8 -> W4.4 -> W4.6

The bottleneck is Wave 3 (dispatch layer) which is sequential and owned by Tech Lead. Waves 2's integrator work parallelizes well (A and B work simultaneously). DX/UX audits run after each wave's code changes complete, in parallel with test work.


Panel Review Protocol

Each wave ends with a structured review:

  1. Code Reviewer runs code-review agent on the wave's diff — surfaces bugs, logic errors, security issues only (no style nits)
  2. DX/UX Expert (via cli-logging-ux skill) audits:
    • Every _rich_* call, click.echo, console message in changed code
    • Status symbols follow STATUS_SYMBOLS convention ([+], [!], [x], [i], [*], [>])
    • Messages are actionable: tell the user what happened AND what to do about it
    • Verbose output adds per-target/per-primitive detail, default output stays clean
    • Error paths include contextual info (which target, which file, which package)
    • Diagnostic summaries (DiagnosticCollector) capture enough for debugging
    • No regressions in existing CLI output quality
  3. Tech Lead validates:
    • Method signatures match the contract from W1.1
    • No cross-target side effects (target A's integration doesn't touch target B's dirs)
    • Performance: no O(N x M) regressions (managed_files lookups stay O(1))
  4. Test Engineer confirms:
    • All existing tests pass without modification OR are explicitly migrated
    • New tests cover the specific scenarios from the test strategy
    • No test is deleted without replacement

Issues found in review are fixed before the next wave starts.


Detailed Phase Descriptions

Phase 1: CommandIntegrator (smallest, proves the pattern)

Current state: 2 integrate methods + 2 sync methods, 95% identical (only path strings and opt-in guard differ).

command_integrator.py:

  • Add integrate_commands_for_target(self, target: TargetProfile, package_info, project_root, *, force=False, managed_files=None, diagnostics=None) -> IntegrationResult
    • Reads target.root_dir and target.primitives["commands"].subdir to compute deploy dir
    • Uses target.detect_by_dir for opt-in guard (replaces hardcoded .opencode existence check)
    • Extension from PrimitiveMapping.extension
  • Add sync_for_target(self, target: TargetProfile, primitive: str, apm_package, project_root, managed_files=None) -> Dict
    • Computes prefix from target.root_dir + "/" + mapping.subdir + "/"
    • Computes legacy glob from mapping.extension
  • Delete old per-target methods (integrate_package_commands, integrate_package_commands_opencode, sync_integration, sync_integration_opencode)

Files touched: command_integrator.py
Lines deleted: ~100
Lines added: ~50


Phase 2: AgentIntegrator (biggest payoff)

Current state: 4 integrate methods + 4 sync methods + 3 get_target_filename methods. ~400 lines, 85-90% identical.

Critical nuance: integrate_package_agents() currently deploys to .github AND .claude AND .cursor in one call (multi-target baked in). This must be split: each call handles ONE target, called once per active target by the dispatch loop.

agent_integrator.py:

  • Add get_target_filename_for_target(self, source_file, package_name, target: TargetProfile) -> str
    • Reads PrimitiveMapping.extension to decide .agent.md vs .md
    • Replaces 3 separate get_target_filename methods
  • Add integrate_agents_for_target(self, target: TargetProfile, package_info, project_root, *, force=False, managed_files=None, diagnostics=None) -> IntegrationResult
    • Single method handles all targets
    • Uses target.auto_create to decide whether to create dir
    • Uses target.detect_by_dir for opt-in guard
    • NO multi-target logic inside — called once per target
  • Add sync_for_target(...) — same pattern
  • Delete old per-target methods

Files touched: agent_integrator.py
Lines deleted: ~350
Lines added: ~70


Phase 3: InstructionIntegrator

Current state: 2 integrate methods + 2 sync methods. 75% identical but with a genuine content transform for Cursor (.mdc format with frontmatter remapping).

instruction_integrator.py:

  • Add integrate_instructions_for_target(self, target: TargetProfile, package_info, project_root, *, force=False, managed_files=None, diagnostics=None) -> IntegrationResult
    • Uses PrimitiveMapping.format_id to select content transform:
      • "github_instructions" -> identity (copy_instruction)
      • "cursor_rules" -> apply _convert_to_cursor_rules transform (copy_instruction_cursor)
    • Uses PrimitiveMapping.extension for target filename
    • Uses PrimitiveMapping.subdir for target dir
  • Add sync_for_target(...) — same pattern
  • _convert_to_cursor_rules() stays as-is — it's the genuine transform logic
  • Delete old per-target methods

First real consumer of format_id — this phase proves the format-driven transform pattern.

Files touched: instruction_integrator.py
Lines deleted: ~80
Lines added: ~60


Phase 4: CLI Dispatch Cleanup (install.py + uninstall/engine.py)

Current state: install.py uses _targets from active_targets() then IGNORES it, computing boolean flags via should_integrate_vscode/claude/opencode. The ~200-line if-chain calls per-target methods individually.

install.py:

  • Delete should_integrate_vscode/claude/opencode imports and flag computation
  • Replace _integrate_package_primitives() body with target-driven loop + PRIMITIVE_TO_INTEGRATOR dict (5 entries)
  • Skills keep existing multi-target handling (SkillIntegrator already does this)
  • Pass targets=_targets instead of boolean flags

uninstall/engine.py:

  • Replace per-integrator sync calls with target-driven loop
  • Replace partition_managed_files() call with simpler per-target filtering

target_detection.py:

  • Delete should_integrate_vscode(), should_integrate_claude(), should_integrate_opencode() — dead code

Files touched: install.py, uninstall/engine.py, target_detection.py
Lines deleted: ~320
Lines added: ~60


Phase 5: Derive partition_managed_files from TargetProfile

Current state: partition_managed_files() has 11 hardcoded buckets with hardcoded prefix strings.

base_integrator.py:

  • Replace hardcoded buckets with dynamic generation from KNOWN_TARGETS
  • Skills and hooks retain cross-target grouping (they handle multi-target internally)
  • Update _sync_integrations_after_uninstall() to use new partitioning

Files touched: base_integrator.py, uninstall/engine.py
Lines deleted: ~50
Lines added: ~30


Phase 6: Final Cleanup

  • Delete old per-target method stubs if any remain
  • Update __init__.py exports
  • Remove unused imports

HookIntegrator: Intentionally Deferred

HookIntegrator has genuine algorithmic diversity (file-per-hook vs merge-into-settings-JSON). Its three integrate methods share ~65% code but diverge on the write strategy.

What we DO in Wave 2: Add a thin integrate_hooks_for_target(target, ...) method that dispatches to existing per-target methods based on target.name. This gives the uniform interface without forcing the algorithmic refactor.

Future: If a 4th hook format arrives, extract shared boilerplate and add a write-strategy callback. Not before.


Test Strategy

Existing test baseline (must all pass)

534 tests across 15 files:

  • Agent integrator: 91 tests (5 classes covering all 4 targets + naming + sync)
  • Hook integrator: 84 tests (9 classes covering VSCode/Claude/Cursor + sync + e2e)
  • Skill integrator: 116 tests (13 classes — already data-driven, should need NO changes)
  • Instruction integrator: 35 tests (collision, sync, Cursor rules)
  • Command integrator: 21 tests (sync, OpenCode, target gating)
  • Prompt integrator: 22 tests (discovery, sync, naming)
  • Deployed files manifest: 69 tests (collision detection for all primitives)
  • Targets: 10 tests (active_targets detection + fallback)
  • Install/uninstall e2e: 44 tests
  • Other: 42 tests

New tests to add

A. Target-driven integration tests (per integrator):
For each refactored integrator, test integrate_*_for_target() with:

  • Each real TargetProfile from KNOWN_TARGETS that supports the primitive
  • Synthetic TargetProfile (proves adding a new target needs no code changes)
  • detect_by_dir=True target when dir doesn't exist (should skip)
  • auto_create=True vs auto_create=False behavior
  • Collision detection with managed_files via target-driven path

B. Target-gating regression tests:

  • test_opencode_only_skips_github_dirs — opencode target does NOT write to .github/
  • test_cursor_only_skips_claude_and_github — cursor target writes only to .cursor/
  • test_copilot_only_skips_cursor_opencode — copilot target writes only to .github/
  • test_empty_targets_returns_zeros — no targets → no integration
  • test_all_targets_hits_every_primitive — with all 4 targets, every (target,primitive) fires

C. Dispatch loop tests:

  • Test _integrate_package_primitives() with targets=[KNOWN_TARGETS["opencode"]] — verify only opencode methods fire
  • Test with targets=list(KNOWN_TARGETS.values()) — verify complete coverage
  • Test that skills are still handled via SkillIntegrator's multi-target path

D. Sync tests:

  • sync_for_target() computes correct prefix from TargetProfile
  • Partition_managed_files dynamic generation matches old hardcoded buckets (transitional parity test)
  • Uninstall flow with target-driven sync removes correct files

E. Exhaustiveness / structural tests:

  • test_every_target_primitive_has_integrator_path — for each (target, primitive) in KNOWN_TARGETS, verify the dispatch loop would route to a real integrator with a working method
  • test_format_id_handlers_cover_all_format_ids — every format_id in KNOWN_TARGETS has a handler

F. Existing test migration:

  • Update all 21 command_integrator tests to use new API
  • Update all 91 agent_integrator tests — particularly the 4 target-specific test classes
  • Update all 35 instruction_integrator tests
  • 84 hook_integrator tests should need NO changes (HookIntegrator keeps per-target methods internally)
  • 116 skill_integrator tests should need NO changes (already data-driven)

Test execution

  • After each wave: run targeted test files
  • Before shipping: full unit suite (uv run pytest tests/unit tests/test_console.py -x)

Risk Mitigation

  • All-or-nothing ship: Everything lands in one PR, avoiding broken intermediate states in main
  • integrate_package_agents multi-target split (Phase 2) is the riskiest change — it changes from "one call deploys to 3 targets" to "3 calls, one per target". Target-gating regression tests are the safety net
  • Existing tests as safety net: 534 tests must all pass before PR is ready
  • HookIntegrator stays internal: Its per-target methods aren't collapsed, avoiding the riskiest refactor
  • format_id: Only consumed for InstructionIntegrator initially — if the pattern doesn't work cleanly, we can fall back to method-name dispatch without blocking the rest
  • Panel reviews gate each wave: No wave starts until the previous passes review

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions