Skip to content

Pilot: issue-tracker-based agent orchestration (Symphony-inspired) for dev-tools #283

@coisa

Description

@coisa

Problem

Team workflow is still centered on interactive agent sessions and manual command launches, which increases context switching and does not scale when multiple GitHub issue-driven tasks run in parallel.

Proposal

Adopt a Symfony-style issue-orchestration pattern inspired by OpenAI’s Symphony concept, where open GitHub issues become the control plane for agent work:

  • Every eligible issue enters a bounded queue for automated execution.
  • A worker process orchestrates environment prep, runs an agent for each issue in isolation, and posts proof-of-work artifacts.
  • Human review remains the final gate for risky changes.

This should be introduced as a pilot in dev-tools and evaluated for reuse by other Fast Forward repos.

Goals

  • Define a deterministic process where each issue maps to one isolated execution context.
  • Route issue lifecycle through explicit states (queued, running, waiting-review, done, blocked, etc.).
  • Keep outputs machine-consumable and CI-safe (log parsing, command artifacts, deterministic logs).
  • Ensure the agent writes:
    • GitHub issue comments with summary and PR/commit links
    • Required PR updates when code/workflow changes are ready
    • Failure evidence when blocked.

Expected Behavior

  • A maintainer can create a regular issue with a defined objective and have the next orchestration run pick it up automatically.
  • The system performs non-interactive analysis/implementation with retries and bounded concurrency.
  • Complex changes do not block single-threaded manual intervention.
  • Successful runs conclude in a reviewable, auditable state (agent-review + attached PR/comment).

Implementation Strategy

Phase 0 - Feasibility Scaffold (recommended first step)

  • Create a proposal WORKFLOW.md in vendor/fast-forward/dev-tools that defines:
    • task intent
    • acceptance criteria format
    • deterministic command matrix to run (composer dev-tools, focused PHPUnit, lint tasks)
    • stop conditions and blocker rules
  • Add issue intake contract (required title/body fields: objective, acceptance criteria, constraints, failure mode, allowed scope).
  • Implement a thin orchestrator script/workflow that:
    • polls GitHub issues by labels/queries and states,
    • creates per-issue workspaces,
    • launches the agent with repo-specific prompt,
    • writes structured status artifacts.

Phase 1 - Safety + Guardrails

  • Add explicit retry and backoff policy for transient CI/API failures.
  • Add hard guardrails for:
    • destructive command prevention,
    • required metadata completeness,
    • output normalization for downstream tooling,
    • workspace cleanup for failed/blocked runs.
  • Store run results in versioned JSON artifacts for diagnostics.

Phase 2 - Review and Handoff

  • Automatically append issue comments with:
    • what changed,
    • validation results,
    • PR links,
    • blocking reasons (if any).
  • Escalate non-trivial/failed runs to agent-review with explicit call-to-action.

Phase 3 - Rollout Model

  • Start with documentation/workflow/automation tasks only (low risk).
  • Expand to changelog/chore tasks once baseline reliability is proven.
  • Keep manual override available for any stage.

Integration with current Fast Forward tools

  • Use github-issues skill-style mutations for issue lifecycle operations.
  • Use existing CI skill patterns to validate PRs before merge (github-pull-request, review skill workflows).
  • Keep dev-tools commands as thin orchestration surfaces:
    • avoid hard-coding tool policy in ad-hoc commands,
    • reuse common process/error primitives,
    • keep deterministic execution order and artifact names.

Non-goals

  • Full replacement of human engineering judgment or review.
  • Unlimited autonomous multi-project self-editing without explicit repository consent.
  • Ingestion of arbitrary issue types beyond clearly scoped repository maintenance tasks.
  • Introducing external issue tracker vendors before github/issues path is proven.

Benefits

  • Lower cognitive switching: less manual steering per issue, more standardized throughput.
  • Better throughput for repetitive, deterministic DevTools workflow improvements.
  • Centralized audit trail from issue → execution → artifacts → PR.
  • Better separation between strategy (issue + WORKFLOW.md) and execution.

Acceptance Criteria

Functional Criteria

  • A repository issue matching configured criteria is automatically moved through a deterministic run pipeline.
  • The system creates isolated execution context per issue.
  • Success and failure outcomes are posted back to the originating issue.
  • A successful execution results in a PR or explicit human-review handoff state.
  • Existing manual issue triage is not replaced; it is supplemented by orchestrated runs.
  • Existing CI-critical workflows continue to be represented by review/status-safe PR flow.

Architectural / Isolation Criteria

  • MUST: The core logic MUST be isolated into dedicated classes or services instead of living inside command or controller entrypoints.
  • MUST: Responsibilities MUST be separated across input resolution, domain logic, processing or transformation, and output rendering when the change is non-trivial.
  • MUST: The command or controller layer MUST act only as an orchestrator.
  • MUST: The implementation MUST avoid tight coupling between core behavior and CLI or framework-specific I/O.
  • MUST: The design MUST allow future extraction or reuse with minimal changes.
  • MUST: The solution MUST remain extensible without requiring major refactoring for adjacent use cases.
  • MUST: Argument and option resolution MUST be validated separately from command execution logic.
  • MUST: Exit behavior, error messaging, and generated output MUST remain deterministic and testable.
  • MUST: Data gathering or transformation MUST be isolated from filesystem writes or publishing steps.
  • MUST: Re-running the orchestration workflow MUST be idempotent or have bounded, explicit side effects.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Merged

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions