Feature: Validate output templates against conditional execution paths

[e-s-gh]

Feature Request

The output: section of a workflow can reference agent variables that may not be defined at runtime due to conditional routing. The validator does not currently detect this — it only checks that route targets exist and all paths reach ``. The result is a runtime TemplateError after the entire workflow has completed, losing all the work done by agents.

Example

evaluator:
  routes:
    - to: deployer
      when: "{{ output.recommendation == 'accept' }}"
    - to: refiner
      when: "{{ output.recommendation == 'revise' }}"
    - to: $end   # reject - deployer never runs

output:
  summary: "{{ deployer.output.summary | default('N/A') }}"   # CRASHES when deployer didn't run

| default() handles undefined attributes (e.g., deployer.output.missing_field) but not undefined variables (the deployer object itself). This is expected Jinja2 StrictUndefined behavior, but easy for workflow authors to get wrong.

Observed Impact

In testing a complex 8-agent workflow (evergreen-docs), the evaluator returned reject and routed to ``. Four agents (dispatcher, generator, reviewer, evaluator) completed successfully over ~42 minutes of execution. The workflow then crashed on the output: template with:

TemplateError: Undefined variable in template: 'deployer' is undefined

All agent work was lost because the final output couldn't be rendered.

What the Validator Currently Checks

• All route targets reference valid agent names, ``, or self
• All agents are reachable from entry_point
• All paths eventually reach ``
• when conditions are syntactically valid Jinja2
• Agent names are unique
• Parallel groups have >= 2 agents
• Output templates reference existing agent names

What It Doesn't Check

1. Output template variables vs execution paths — The output: section may reference agents that are skipped on certain execution paths. The validator doesn't trace conditional routes to determine which agents are guaranteed to run.

2. Agent prompt templates vs execution paths — An agent's prompt might reference {{ other_agent.output.field }} without guards where other_agent only runs on some paths.

3. | default() vs {% if is defined %} correctness — The validator doesn't warn that | default() won't protect against an entirely undefined variable.

Proposed Improvement: Static Path Analysis for Output Templates

The validator already knows the routing graph. It could:

1. Enumerate all possible execution paths from entry_point to ``
2. For each path, determine which agents will have executed
3. Check whether the output: template references any agent that doesn't appear on every path
4. Emit a warning (not error) for agents that are path-dependent

Example output:

Warning: output template references 'deployer' which does not execute on path:
     dispatcher -> generator -> reviewer -> evaluator -> 
   Consider using {% if deployer is defined %} guards.

This should be a warning rather than an error because the workflow author may intend for some output fields to be empty on certain paths.

Additional Ideas

• Safe default syntax: A null-coalescing operator like {{ deployer.output.summary ?? 'N/A' }} to handle both undefined variables and attributes in a single expression
• Optional agent reference marker: Similar to how input: uses ? for optional dependencies, output templates could mark references as intentionally conditional

Current Workaround

Use {% if agent is defined and agent.output %} guards in the output template:

output:
  summary: "{% if deployer is defined and deployer.output %}{{ deployer.output.summary }}{% else %}Workflow ended before deployment{% endif %}"

Priority

Medium-high. This is a workflow-author footgun that causes failures after long-running workflows complete. The validator has enough information (the routing graph) to detect this statically in most cases.