Skip to content

feat(composition): allow sub-workflows in for_each groups #102

@PolyphonyRequiem

Description

@PolyphonyRequiem

Provenance: AI-drafted (GitHub Copilot CLI / Claude Opus 4.6), human-reviewed and approved, AI-submitted.

Problem

for_each groups currently reject type: workflow agents:

# src/conductor/config/validator.py
if for_each_group.agent.type == "workflow":
    errors.append(
        f"For-each group '{for_each_group.name}' uses a workflow step..."
    )

This prevents dynamic fan-out to sub-workflows. A common pattern — "for each issue in the plan, run a planning/implementation sub-workflow" — is impossible.

Proposed solution

Remove the validator restriction and wire up for_each execution to call _execute_subworkflow() for type: workflow agents.

agents:
  - name: epic_planner
    prompt: "Break this epic into issues"
    output:
      issues: { type: array }
    routes:
      - to: plan_issues

  - name: plan_issues
    type: for_each
    source: epic_planner.output.issues
    as: issue
    max_concurrent: 1   # sequential to avoid branch conflicts
    agent:
      type: workflow
      workflow: ./plan-and-review.yaml
      input_mapping:                        # from #101
        work_item_id: "{{ issue.id }}"
        title: "{{ issue.title }}"
    routes:
      - to: next_step

Key design decisions:

  • max_concurrent should default to 1 for workflow steps (sub-workflows may have side effects — git, file system — that conflict when parallel)
  • Each for_each iteration receives the item/as variable in scope for input_mapping template rendering
  • Sub-workflow depth increments per nesting level, shared across all for_each iterations (prevents depth explosion)
  • Failure mode (fail_fast vs continue) applies as with regular agents

Engine change:
In _execute_for_each_group(), when agent.type == "workflow", call _execute_subworkflow(agent, item_context) instead of the regular agent execution path. The input_mapping templates are rendered with the for_each item variable in scope.

Dependency

Requires #101 (input_mapping) — without dynamic inputs, every for_each iteration would receive identical workflow.input.*, defeating the purpose.

Relationship to other requests

This is Issue 2 of 3 in a series enabling recursive workflow composition:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions