-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Summary
The pipeline template engine uses Option("missingkey=zero") (Go text/template), which means any typo in a field-level reference silently resolves to an empty/zero value at runtime. There is no compile-time, load-time, or static-analysis check that catches these errors.
Combined with the fact that all inter-step data flows through map[string]any (no Go structs), this creates a class of bugs that are invisible until runtime — and even then may not produce obvious errors, just silently wrong data.
Current behavior
steps:
- name: auth
type: step.auth_validate
- name: process
type: step.set
config:
values:
# Correct: resolves to the actual affiliate_id
tenant: "{{ .steps.auth.affiliate_id }}"
# Typo: silently resolves to "" — no error at any stage
role: "{{ .steps.auth.affilate_id }}"rolesilently becomes""at runtime- No error in pipeline execution (pipeline "succeeds")
- Downstream steps receive empty string instead of failing fast
wfctl template validatecatches step name typos (steps.nonexistent_step) but NOT field-level typos (steps.auth.misspelled_field)
Root cause
PipelineContextstores all data asmap[string]map[string]any(interfaces/pipeline.go:51-52) — no struct fields to catch at compile timeTemplateEngine.Resolve()uses.Option("missingkey=zero")(module/pipeline_template.go:355) — Go'stext/templatereturns the zero value for missing map keys instead of erroringwfctl template validateregex\.steps\.([a-zA-Z_][a-zA-Z0-9_-]*)only extracts the step name portion, completely ignoring the field path after it (cmd/wfctl/template_validate.go:569)
Impact
This affects every pipeline that references step outputs. Common failure modes:
- Typo in field name:
steps.auth.affilate_id→ empty string, pipeline proceeds with wrong data - Rename a step output field: Downstream steps silently get empty instead of failing
- Nested object access:
steps.query.row.column_name— no validation thatrowcontainscolumn_name - Conditional guards:
if: "{{ .steps.decode.valid }}"— a typo here means the guard always evaluates one way
Proposed solutions
Option A: missingkey=error mode (strict templates)
Add an opt-in strict mode that switches to Option("missingkey=error"):
pipelines:
my-pipeline:
strict_templates: true # or a global engine settingThis would cause text/template to return an error instead of zero when a key is missing. Breaking change if enabled globally, but could be opt-in per pipeline or via a wfctl flag.
Option B: Enhanced static validation in wfctl template validate
Extend validateStepRef() to infer expected output schemas from step types:
step.db_querywithmode: single→ output hasrow,foundstep.db_querywithmode: list→ output hasrows,countstep.auth_validate→ output hasauth_user_id,affiliate_idstep.request_parse→ output hasheaders,body,path_params,query_paramsstep.base64_decode→ output hasvalid,data,mime_type, etc.
Then validate that field-level references in templates match the known output schema of the referenced step.
Option C: Runtime warning mode
Log a warning (not error) when a template resolves a missing key to zero value. This preserves backward compatibility while making silent failures visible:
WARN template resolved missing key: steps.auth.affilate_id → "" (pipeline: my-pipeline, step: process)
References
- Template resolution:
module/pipeline_template.go:348-366(missingkey=zeroon line 355) - Pipeline context:
interfaces/pipeline.go:46-60(map[string]anytypes) - Step output merging:
interfaces/pipeline.go:87-98 - Template validation:
cmd/wfctl/template_validate.go:568-646(step name only) - Test proving silent zero:
module/pipeline_template_test.go:93-104(TestTemplateEngine_MissingKeyReturnsZeroValue)