test: add --help scenario tests for cut, echo, head, printf, sed, sort, tail, wc#151
test: add --help scenario tests for cut, echo, head, printf, sed, sort, tail, wc#151
Conversation
…t, tail, wc These 8 commands were missing scenario tests for their --help/-h flags. Each test verifies: exit code 0 (or 2 for printf), usage printed to stdout, no stderr output. Also documents that echo treats --help as literal text. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
@codex review this PR |
thieman
left a comment
There was a problem hiding this comment.
Self-Review: PR #151 — Add help flag scenario tests
Scope reviewed: 12 new YAML scenario test files across 8 commands (cut, echo, head, printf, sed, sort, tail, wc).
Overall assessment: needs minor fixes — the tests correctly capture help-flag behavior and all pass. Two findings below.
Coverage Summary
| Code path | Scenario test | Go test | Status |
|---|---|---|---|
cut --help → stdout, exit 0 |
cut/help/help_flag.yaml |
— | Covered |
echo --help → prints literal text, exit 0 |
echo/help/help_flag.yaml |
— | Covered |
head --help → stdout, exit 0 |
head/help/help_flag.yaml |
— | Covered |
head -h → stdout, exit 0 |
head/help/short_help_flag.yaml |
— | Covered |
printf --help → stdout, exit 2 |
printf/help/help_flag.yaml |
— | Covered |
sed --help → stdout, exit 0 |
sed/help/help_flag.yaml |
— | Covered |
sed -h → stdout, exit 0 |
sed/help/short_help_flag.yaml |
— | Covered |
sort --help → stdout, exit 0 |
sort/help/help_flag.yaml |
— | Covered |
sort -h → stdout, exit 0 |
sort/help/short_help_flag.yaml |
— | Covered |
tail --help → stdout, exit 0 |
tail/help/help_flag.yaml |
— | Covered |
tail -h → stdout, exit 0 |
tail/help/short_help_flag.yaml |
— | Covered |
wc --help → stdout, exit 0 |
wc/help/help_flag.yaml |
— | Covered |
Findings
Positive Observations
- All 12 scenarios correctly align with the actual builtin implementations (verified by reading source and running tests).
exit_code: 2forprintf --helpcorrectly matches the implementation.- Commands without a
-hshort flag (cut,wc) correctly omitshort_help_flag.yaml. - All files follow the project YAML conventions (
|+block scalars,expect.stderrexact match).
|
Iteration 1 self-review result: 2 findings — 1× P2, 1× P3. No security or correctness issues. The P2 is an unnecessary |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bfee7e45e4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,11 @@ | |||
| description: echo --help is not recognised as a flag and is printed as literal text. | |||
| # skip: echo does not implement --help; bash behaves the same way | |||
| skip_assert_against_bash: true | |||
There was a problem hiding this comment.
Re-enable bash comparison for this echo --help scenario
This test is explicitly asserting bash-equivalent behavior (echo --help is treated as a literal string), but skip_assert_against_bash: true prevents it from running in TestShellScenariosAgainstBash. I verified in bash that echo --help exits 0, writes --help to stdout, and leaves stderr empty, so the current expectations are compatible; skipping here removes useful compatibility coverage and can hide regressions.
Useful? React with 👍 / 👎.
- tests/scenarios/cmd/echo/help/help_flag.yaml: Remove skip_assert_against_bash: true since bash echo --help behaves identically to rshell (prints "--help", exits 0). The test now validates against bash as it should. - tests/scenarios/cmd/printf/help/help_flag.yaml: Strengthen stdout_contains from ["printf"] to ["printf: usage: printf"] to match the actual help string and avoid loose matching. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Problem
T-1: Missing --help scenario tests for 8 commands
RULES.md requires: "Every supported flag MUST have correctness tests."
These 8 commands had no scenario tests verifying that
--help(and-hwhere supported) exits with the correct code and prints usage to stdout:cut—--helponlyecho— no--helpflag; documents that it prints the literal stringhead—--helpand-hprintf—--helponly (exits 2 per implementation)sed—--helpand-hsort—--helpand-htail—--helpand-hwc—--helponlyFix
Added
help/help_flag.yaml(andhelp/short_help_flag.yamlfor-hvariants) for each command. Tests verify:printfwhich exits 2)stdout_contains)All tests are marked
skip_assert_against_bash: truesince bash handles these differently.Test plan
go test ./tests/... -run TestShellScenariospasses with all 12 new scenario files🤖 Generated with Claude Code