Pipeline and step output type schemas are metadata-only — no runtime or static enforcement

[intel352]

Summary

The engine has comprehensive type metadata for step outputs (182 built-in steps declare their output keys, types, and descriptions via schema.StepOutputDef), but this metadata is never enforced — not at config load time, not at runtime, and not by wfctl validate. The schemas serve documentation purposes only.

This is a separate concern from #367 (missingkey=zero for template resolution). Even if #367 is fixed, there is no mechanism to declare or validate the shape of data flowing between steps or returned from pipelines.

What exists today (metadata, not enforcement)

Every built-in step declares its outputs in schema/step_schema_builtins.go:

// step.db_query (mode: single)
Outputs: []StepOutputDef{
    {Key: "row",   Type: "map",     Description: "First result row..."},
    {Key: "found", Type: "boolean", Description: "Whether a row was found"},
}

// step.auth_validate
Outputs: []StepOutputDef{
    {Key: "(claims)",       Type: "any",    Description: "All claims..."},
    {Key: "(subject_field)", Type: "string", Description: "Value of sub claim..."},
}

These are exposed via wfctl get_step_schema and MCP tooling. But at runtime, StepResult.Output is always map[string]any — the engine never checks that the actual output matches the declared schema.

What's missing

1. Pipeline-level output contract

PipelineConfig has no outputs: field:

type PipelineConfig struct {
    Trigger      PipelineTriggerConfig
    Steps        []PipelineStepConfig
    OnError      string
    Timeout      string
    Compensation []PipelineStepConfig
    // ← No OutputSchema, no Outputs, no ResponseContract
}

A pipeline can return anything. Callers (HTTP triggers, step.workflow_call, eventbus consumers) have no declared contract for what they'll receive.

2. Step output validation

When a step completes, the engine merges its output into PipelineContext.Current without checking that the output matches the step type's declared schema:

// interfaces/pipeline.go:87-98
func (pc *PipelineContext) MergeStepOutput(stepName string, output map[string]any) {
    // No validation that output matches StepSchema.Outputs
    pc.StepOutputs[stepName] = stepOut
    maps.Copy(pc.Current, output)
}

3. Cross-step type compatibility in wfctl

The validator checks that step names exist but never checks that referenced fields exist in the step type's declared output schema. The schema metadata is already there — it just isn't used.

Example: {{.steps.query.row.slug}} — the validator could check:

1. query step exists ✅ (already done)
2. query is step.db_query → declared outputs include row (type: map) — could do, not done
3. row contains slug — can't know (depends on SQL), but could warn it's unverifiable

Even checking (2) would catch bugs like referencing steps.query.rows when the step uses mode: single (which only outputs row + found, not rows + count).

4. step.workflow_call output is (dynamic): any

Callers get no type contract from the called pipeline. If the called pipeline declared its output schema, step.workflow_call could validate that output_mapping references valid fields.

5. wfctl contract test has no response schemas

EndpointContract captures method, path, pipeline name — but no request/response schema. Output field removals or type changes are invisible to contract diffing.

Suggested implementation (backwards-compatible)

All proposals are opt-in — existing configs continue to work unchanged.

Phase 1: Static analysis with existing metadata (no config changes)

Use the existing StepSchema.Outputs in wfctl template validate to warn when a template references a field not in the step type's declared output. This requires zero config changes — the metadata already exists for all 182 built-in steps.

$ wfctl template validate --config pipeline.yaml
WARNING: pipeline "get-form" step "respond": references steps.query.rows
         but step "query" (step.db_query, mode: single) outputs: row, found
         (did you mean "row"?)

Phase 2: Optional pipeline output declarations

Add an optional outputs: block to pipeline config. When absent, behavior is unchanged (fully backwards-compatible). When present, enables downstream validation.

pipelines:
  get-form:
    outputs:
      id:
        type: string
      slug:
        type: string
      found:
        type: boolean
    steps: [...]

Benefits:

• step.workflow_call callers can validate output_mapping against the called pipeline's declared outputs
• wfctl contract test can include response schemas in endpoint contracts
• Serves as inline documentation of the pipeline's data contract

Phase 3: Optional runtime validation

Add an opt-in strict_outputs flag (per-pipeline or global). When enabled, the engine validates step output against declared schemas at execution time. When disabled (default), behavior is unchanged.

pipelines:
  get-form:
    strict_outputs: true  # opt-in runtime validation
    outputs:
      id: { type: string }
    steps: [...]

Or as a global engine setting:

engine:
  strict_outputs: warn  # "off" (default), "warn", "error"

Phase 4: Contract enrichment

Include pipeline output schemas in EndpointContract so wfctl contract test can detect breaking changes when output fields are removed or change type:

type EndpointContract struct {
    Method         string            `json:"method"`
    Path           string            `json:"path"`
    Pipeline       string            `json:"pipeline"`
    ResponseSchema map[string]string `json:"responseSchema,omitempty"` // field → type
}

Impact without this

• Refactoring risk: Renaming a step output field silently breaks all downstream consumers
• Cross-pipeline risk: step.workflow_call callers have no way to know what the called pipeline returns
• API stability risk: HTTP response shapes can change without detection by wfctl contract test
• Wasted metadata: 182 step types declare output schemas that are never used for validation

References

• Step output schema metadata: schema/step_schema.go (StepOutputDef)
• Built-in schemas: schema/step_schema_builtins.go (182 step types)
• Pipeline context (untyped): interfaces/pipeline.go:46-60
• Output merging (no validation): interfaces/pipeline.go:87-98
• Contract generation: cmd/wfctl/contract.go (no schema fields)
• Template validation: cmd/wfctl/template_validate.go (step name only)
• Related: Template missingkey=zero silently swallows field-level typos in step references #367 (missingkey=zero), wfctl validate: no cross-reference checking for DB columns, step output fields, or imported configs #368 (no cross-reference validation)