wftest YAML runner: add JSON path and header response assertions#370
Merged
wftest YAML runner: add JSON path and header response assertions#370
Conversation
…nner Co-authored-by: intel352 <77607+intel352@users.noreply.github.com> Agent-Logs-Url: https://github.com/GoCodeAlone/workflow/sessions/50829bd6-f7d6-427c-be25-315c7c72e09a
Copilot
AI
changed the title
[WIP] Extend ResponseAssert to support JSON path and header assertions
wftest YAML runner: add JSON path and header response assertions
Mar 24, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Adds richer HTTP response assertions to the wftest YAML runner so YAML tests can assert specific JSON fields and response headers (parity with existing BDD assertions), rather than relying only on status and body substring checks.
Changes:
- Extend
ResponseAssertYAML schema withjson,json_not_empty, andheaders. - Implement evaluation logic for these new assertions in the YAML runner, including a dot-path JSON traversal helper.
- Add a YAML runner test that exercises JSON path equality, JSON-not-empty, and header assertions.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
wftest/yaml_types.go |
Adds new YAML fields to ResponseAssert to express JSON-path and header expectations. |
wftest/yaml_runner.go |
Implements the new assertion behaviors and adds a jsonPath helper. |
wftest/yaml_runner_test.go |
Adds a new test YAML scenario validating JSON path + non-empty + header assertions. |
wftest/yaml_runner.go
Outdated
Comment on lines
+243
to
+245
| if fmt.Sprintf("%v", val) != fmt.Sprintf("%v", expected) { | ||
| t.Errorf("assertion %s: JSON %q: want %v, got %v", label, path, expected, val) | ||
| } |
There was a problem hiding this comment.
Response.JSON assertions compare val and expected via fmt.Sprintf, which becomes unstable/incorrect for non-scalar JSON values (e.g., objects) because Go map iteration order is randomized and the string form can differ even when the JSON structures are equivalent. Consider comparing by marshaling both sides to JSON (or using a deep-equality that normalizes numeric types) to support exact matches reliably for nested values.
Suggested change
| if fmt.Sprintf("%v", val) != fmt.Sprintf("%v", expected) { | |
| t.Errorf("assertion %s: JSON %q: want %v, got %v", label, path, expected, val) | |
| } | |
| wantJSON, err := json.Marshal(expected) | |
| if err != nil { | |
| t.Errorf("assertion %s: JSON %q: cannot marshal expected value: %v", label, path, err) | |
| continue | |
| } | |
| gotJSON, err := json.Marshal(val) | |
| if err != nil { | |
| t.Errorf("assertion %s: JSON %q: cannot marshal actual value: %v", label, path, err) | |
| continue | |
| } | |
| if !bytes.Equal(wantJSON, gotJSON) { | |
| t.Errorf("assertion %s: JSON %q: want %s, got %s", label, path, string(wantJSON), string(gotJSON)) | |
| } |
Comment on lines
+247
to
+255
| for _, path := range a.Response.JSONNotEmpty { | ||
| val, err := jsonPath(result.RawBody, path) | ||
| if err != nil { | ||
| t.Errorf("assertion %s: %v", label, err) | ||
| continue | ||
| } | ||
| if val == nil || fmt.Sprintf("%v", val) == "" { | ||
| t.Errorf("assertion %s: JSON %q: expected non-empty, got %v", label, path, val) | ||
| } |
There was a problem hiding this comment.
json_not_empty currently treats empty arrays/maps as non-empty because it only checks val == nil or fmt.Sprintf("%v", val) == "". This will incorrectly pass for {} and []. Consider handling common JSON container types explicitly (string/[]any/map[string]any) and checking length == 0 as empty.
Comment on lines
+257
to
+261
| for header, expected := range a.Response.Headers { | ||
| actual := result.Header(header) | ||
| if actual != expected { | ||
| t.Errorf("assertion %s: header %q: want %q, got %q", label, header, expected, actual) | ||
| } |
There was a problem hiding this comment.
Header assertions use result.Header(header) which does a direct map lookup; this makes header matching case-sensitive even though HTTP header names are case-insensitive. Consider canonicalizing the asserted header name (e.g., via http.CanonicalHeaderKey) before lookup, or updating Result.Header to do case-insensitive matching.
wftest/yaml_runner.go
Outdated
Comment on lines
+318
to
+337
| // jsonPath traverses a JSON body using a dot-separated path (e.g., "user.name"). | ||
| // Returns the value at the path, or an error if the path cannot be traversed. | ||
| func jsonPath(body []byte, path string) (any, error) { | ||
| var root any | ||
| if err := json.Unmarshal(body, &root); err != nil { | ||
| return nil, fmt.Errorf("JSON path %q: invalid JSON body: %w", path, err) | ||
| } | ||
| parts := strings.Split(path, ".") | ||
| current := root | ||
| for _, part := range parts { | ||
| m, ok := current.(map[string]any) | ||
| if !ok { | ||
| return nil, fmt.Errorf("JSON path %q: cannot traverse into non-object at %q", path, part) | ||
| } | ||
| current, ok = m[part] | ||
| if !ok { | ||
| return nil, fmt.Errorf("JSON path %q: key %q not found", path, part) | ||
| } | ||
| } | ||
| return current, nil |
There was a problem hiding this comment.
jsonPath is now duplicated in both wftest/yaml_runner.go and wftest/bdd/steps_assert.go, which increases the risk of behavior drift if one implementation changes. Consider extracting it into a shared helper in the wftest package and reusing it from both places (and optionally parsing the JSON body once per response assertion block to avoid repeated unmarshalling).
Comment on lines
+249
to
+252
| trigger: | ||
| type: http | ||
| method: GET | ||
| path: /hello |
There was a problem hiding this comment.
This test YAML sets trigger.method: GET, but the YAML runner’s fireTrigger path for type: http currently ignores method and always issues a GET. Consider removing method here to avoid implying it’s honored (or switch to type: http.get/get if the intent is to test GET semantics).
⏱ Benchmark Results✅ No significant performance regressions detected. benchstat comparison (baseline → PR)
|
Contributor
|
@copilot apply changes based on the comments in this thread |
Co-authored-by: intel352 <77607+intel352@users.noreply.github.com> Agent-Logs-Url: https://github.com/GoCodeAlone/workflow/sessions/90bb1450-79b1-4df8-83d0-c3437c764344
Contributor
Author
Applied all 5 review changes in commit 83854ff:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ResponseAssertonly supportedstatusandbody(substring), making it impossible to assert specific JSON keys or response headers in YAML tests.Changes
wftest/yaml_types.go— Added three fields toResponseAssert:json: dot-path → expected value (exact match)json_not_empty: list of dot-paths that must be present and non-emptyheaders: header name → expected valuewftest/yaml_runner.go— ExtendedapplyAssertionto evaluate the new fields; added ajsonPathhelper (dot-separated traversal into unmarshalled JSON, mirrorswftest/bdd/steps_assert.go)wftest/yaml_runner_test.go— AddedTestYAMLRunner_ResponseJSONexercising all three new assertion typesExample
Original prompt
⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.