Skip to content

Pilot: issue-tracker-based agent orchestration (Symphony-inspired) for dev-tools #285

@coisa

Description

@coisa

Problem

Team workflow is still centered on interactive agent sessions and manual command launches, which increases context switching and does not scale when multiple GitHub issue-driven tasks run in parallel.

Proposal

Adopt a GitHub-issue orchestration pattern inspired by OpenAI's Symphony concept, where open issues become the control plane for agent work:

  • Every eligible issue enters a bounded queue for automated execution.
  • A worker process orchestrates environment preparation, runs one agent per issue in isolation, and posts proof-of-work artifacts.
  • Human review remains the final gate for risky changes.

This should start as a pilot in dev-tools and be evaluated for reuse by other Fast Forward projects.

Goals

  • Define a deterministic process where each issue maps to one isolated execution context.
  • Route issue lifecycle through explicit states (queued, running, waiting-review, done, blocked).
  • Keep outputs machine-consumable and CI-safe, with deterministic logs and artifacts.
  • Ensure the agent writes:
    • GitHub issue comments with summary and PR/commit links
    • Required PR updates when code/workflow changes are ready
    • Failure evidence when blocked.

Expected Behavior

  • A maintainer can create a regular issue with a clear objective and have the next orchestration run pick it up automatically.
  • The system performs non-interactive analysis/implementation with retries and bounded concurrency.
  • Complex changes do not block single-threaded manual intervention.
  • Successful runs conclude in a reviewable state (agent-review) with attached PR/comment.

Implementation Strategy

Phase 0 - Feasibility Scaffold

  • Create a repository-specific WORKFLOW.md contract that defines:
    • task intent
    • acceptance criteria format
    • deterministic command matrix (composer dev-tools, focused PHPUnit, lint tasks)
    • stop conditions and blocker rules.
  • Define issue intake requirements in the issue body: objective, acceptance criteria, constraints, failure mode, allowed scope.
  • Add a thin orchestrator that:
    • polls GitHub issues by query/state,
    • creates per-issue workspaces,
    • runs an agent with repo-specific prompt,
    • writes structured status artifacts.

Phase 1 - Safety and Guardrails

  • Add retry and backoff for transient CI/API failures.
  • Add guardrails for:
    • destructive command prevention
    • required metadata completeness
    • output normalization for downstream tooling
    • workspace cleanup for failed or blocked runs
  • Persist run summaries in versioned JSON artifacts for diagnostics.

Phase 2 - Review and Handoff

  • Automatically post issue comments with:
    • what changed
    • validation results
    • PR links
    • blocking reasons when applicable
  • Escalate non-trivial or blocked runs to agent-review with a clear call-to-action.

Phase 3 - Rollout

  • Start with documentation/workflow tasks only.
  • Expand to changelog/chore workflows after baseline reliability.
  • Keep manual override available at every stage.

Integration with existing tools

  • Keep issue lifecycle actions aligned with existing Fast Forward GitHub workflows.
  • Use dedicated skills for issue creation/updates, PR publication, and review automation.
  • Keep dev-tools command surfaces thin and deterministic, with clear separation between orchestration and domain logic.

Non-goals

  • Replace human engineering judgment or final PR review.
  • Permit autonomous multi-project edits without explicit repository-level constraints.
  • Introduce external trackers before github issue path is proven.
  • Convert issue triage to a fully autonomous behavior without explicit review gates.

Benefits

  • Reduced context switching and less manual steering of repeated tasks.
  • Faster throughput for deterministic maintenance and automation tasks.
  • Centralized audit trail from issue -> execution -> artifacts -> PR.
  • Cleaner separation between strategy (issue + WORKFLOW.md) and execution.

Acceptance Criteria

Functional Criteria

  • A repository issue matching configured criteria is automatically processed by the orchestration run.
  • Each issue executes in an isolated workspace.
  • Success and failure outcomes are posted back on the originating issue.
  • A successful run produces either a PR or a defined human-review handoff state.
  • Existing CI-critical workflows remain represented in review/status-safe flow.
  • Manual issue triage remains available and unblocked.

Architectural / Isolation Criteria

  • MUST: The core logic MUST be isolated into dedicated classes or services instead of living inside command or controller entrypoints.
  • MUST: Responsibilities MUST be separated across input resolution, domain logic, processing or transformation, and output rendering when the change is non-trivial.
  • MUST: The command or controller layer MUST act only as an orchestrator.
  • MUST: The implementation MUST avoid tight coupling between core behavior and CLI or framework-specific I/O.
  • MUST: The design MUST allow future extraction or reuse with minimal changes.
  • MUST: The solution MUST remain extensible without requiring major refactoring for adjacent use cases.
  • MUST: Argument and option resolution MUST be validated separately from command execution logic.
  • MUST: Exit behavior, error messaging, and generated output MUST remain deterministic and testable.
  • MUST: Data gathering or transformation MUST be isolated from filesystem writes or publishing steps.
  • MUST: Re-running the orchestration workflow MUST be idempotent or have bounded, explicit side effects.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Merged

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions