docs: add CLI end-user validation change proposals (cli-val-01 through 06)

openspec/CHANGE_ORDER.md

@@ -169,6 +169,17 @@ These are derived extensions of the same 2026-02-15 plan and are required to ope
 | ai-integration | 02 | ai-integration-02-mcp-server | [#252](https://github.com/nold-ai/specfact-cli/issues/252) | validation-02 |
 | ai-integration | 03 | ai-integration-03-instruction-files | [#253](https://github.com/nold-ai/specfact-cli/issues/253) | ai-integration-01 |
 
+### CLI end-user validation (validation gap plan, 2026-02-19)
+
+| Module | Order | Change folder | GitHub # | Blocked by |
+|--------|-------|---------------|----------|------------|
+| cli-val | 01 | cli-val-01-behavior-contract-standard | [#279](https://github.com/nold-ai/specfact-cli/issues/279) | — |
+| cli-val | 02 | cli-val-02-output-snapshot-stability | [#280](https://github.com/nold-ai/specfact-cli/issues/280) | — |
+| cli-val | 03 | cli-val-03-misuse-safety-proof | [#281](https://github.com/nold-ai/specfact-cli/issues/281) | #279 |
+| cli-val | 04 | cli-val-04-acceptance-test-runner | [#282](https://github.com/nold-ai/specfact-cli/issues/282) | #279, #281 |
+| cli-val | 05 | cli-val-05-ci-integration | [#283](https://github.com/nold-ai/specfact-cli/issues/283) | #280, #282 |
+| cli-val | 06 | cli-val-06-copilot-test-generation | [#284](https://github.com/nold-ai/specfact-cli/issues/284) | #279 (soft: #283) |
+
 ### Integration governance and proof (architecture integration plan, 2026-02-15)
 
 | Module | Order | Change folder | GitHub # | Blocked by |
@@ -252,6 +263,7 @@ One parent issue per module group for grouping. Set **Type** to Epic on the proj
 | Sidecar validation | [Epic] Sidecar validation | [#191](https://github.com/nold-ai/specfact-cli/issues/191) |
 | Bundle mapping | [Epic] Bundle/spec mapping | [#192](https://github.com/nold-ai/specfact-cli/issues/192) |
 | Architecture + Marketplace | [Epic] Architecture (CLI structure, modularity, performance) | [#194](https://github.com/nold-ai/specfact-cli/issues/194) |
+| CLI end-user validation | [Epic] CLI End-User Validation | [#285](https://github.com/nold-ai/specfact-cli/issues/285) |
 
 ---
 
@@ -267,6 +279,12 @@ Dependencies flow left-to-right; a wave may start once all its hard blockers are
   - backlog-core-01 ✅
   - validation-01 ✅, sidecar-01 ✅, bundle-mapper-01 ✅
 
+- **Wave 1.5 — CLI end-user validation** (cross-cutting, parallel to Wave 2+):
+  - cli-val-01, cli-val-02 (no blockers — start immediately after Wave 1)
+  - cli-val-03, cli-val-06 (after cli-val-01)
+  - cli-val-04 (after cli-val-01 + cli-val-03)
+  - cli-val-05 (after cli-val-02 + cli-val-04 — capstone)
+
 - **Wave 2 — Marketplace + backlog module layer** (needs Wave 1):
   - marketplace-01 (needs arch-06)
   - backlog-core-02 (needs backlog-core-01)
@@ -319,6 +337,7 @@ A wave cannot be considered complete until all gate criteria listed for that wav
 
 - Wave 0 gate: Core modular CLI and bridge registry flows remain stable and archived changes are validated.
 - Wave 1 gate: arch-06/07, policy-engine-01, patch-mode-01, backlog-core-01, validation-01 produce passing contract and strict OpenSpec validation. ✅ Completed 2026-02-18.
+- Wave 1.5 gate: CLI behavior contract schema validated, snapshot tests pass for all command groups, black-box acceptance tests prove installed binary works, anti-pattern safety assertions pass for all Wave 1 commands, CI gates enforce all of the above.
 - Wave 2 gate: At least one backlog planning workflow completes with no blocking dependency regressions across backlog-core + marketplace-01.
 - Wave 3 gate: Higher-order backlog workflows and marketplace-02 interoperate without command-group regressions.
 - Wave 4 gate: `ceremony-cockpit-01` aliases resolve and execute against installed modules without fallback failures.

openspec/changes/cli-val-01-behavior-contract-standard/CHANGE_VALIDATION.md

@@ -0,0 +1,93 @@
+# Change Validation: cli-val-01-behavior-contract-standard
+
+- **Validated on (UTC):** 2026-02-19T12:00:00Z
+- **Workflow:** /wf-validate-change (proposal-stage dry-run validation)
+- **Strict command:** `openspec validate cli-val-01-behavior-contract-standard --strict`
+- **Result:** PASS
+
+## Scope Summary
+
+- **New capabilities:** cli-behavior-contracts
+- **Modified capabilities:** none
+- **Declared dependencies:** none (foundation change with no prerequisites)
+- **Downstream dependents:** cli-val-03-misuse-safety-proof, cli-val-04-acceptance-test-runner, cli-val-06-copilot-test-generation
+- **Proposed affected code paths:**
+  - `tests/cli-contracts/schema/cli-scenario.schema.yaml` (new schema file)
+  - `tests/cli-contracts/*.scenarios.yaml` (new pilot scenario files)
+  - `tools/validate_cli_contracts.py` (new validation tool)
+  - `openspec/config.yaml` (extend with CLI behavior contract artifact type)
+  - `docs/` (new contributor page)
+
+## Format Validation
+
+- **proposal.md Format**: PASS
+  - Title format: Correct (`# Change: CLI Behavior Contract Standard`)
+  - Required sections: All present (Why, What Changes, Capabilities, Impact)
+  - "What Changes" format: Correct (uses NEW/EXTEND markers with bullet list)
+  - "Capabilities" section: Present (one capability: `cli-behavior-contracts`)
+  - "Impact" format: Correct (lists Affected specs, Affected code, Integration points, Documentation impact)
+  - Source Tracking section: Present (GitHub Issue #279, repository nold-ai/specfact-cli)
+  - Note: Section heading uses `## Dependencies` instead of the standard `## Impact` subsection for dependency declarations. Dependencies are still clearly stated. Minor deviation, non-blocking.
+- **tasks.md Format**: PASS
+  - Section headers: Correct (hierarchical numbered format: `## 1.`, `## 2.`, ... `## 8.`)
+  - Task format: Correct (`- [ ] 1.1 [Description]`)
+  - Sub-task format: Correct (`- [ ] 1.1.1 [Description]`)
+  - TDD order enforced section: Present at top
+  - Git worktree creation: First task (`## 1.`)
+  - PR creation: Last implementation task (`## 8. Delivery`, task 8.4)
+  - Post-merge cleanup: Present
+  - Quality gate tasks: Present (`## 6.` includes format, type-check, lint, contract-test, smart-test)
+  - Version and changelog tasks: Present (`## 7.`)
+  - Note: No explicit worktree bootstrap pre-flight tasks (hatch env create, smart-test-status, contract-test-status). Minor gap, non-blocking.
+- **specs Format**: PASS
+  - Given/When/Then format: Verified across all 7 scenarios in `specs/cli-behavior-contracts/spec.md`
+  - Three requirements defined: CLI Behavior Contract Schema (3 scenarios), Pilot Scenario Files (3 scenarios), Schema Validation Tool (1 scenario)
+  - All scenarios use GIVEN/WHEN/THEN/AND structure correctly
+- **design.md Format**: PASS
+  - Context section: Present
+  - Goals / Non-Goals: Present (4 goals, 4 non-goals)
+  - Decisions: Present (5 decisions documented)
+  - Risks / Trade-offs: Present (3 risks with mitigations)
+  - Migration Plan: Present (5-step plan)
+  - Schema Design: Present (bonus detail showing YAML structure)
+  - Open Questions: Present (2 questions)
+  - Note: No bridge adapter integration or sequence diagrams documented, but these are not applicable (no multi-repo flows or adapter integration in this change)
+
+## Dependency and Integration Review
+
+- **CHANGE_ORDER.md consistency**: PASS
+  - CHANGE_ORDER.md row: `cli-val | 01 | cli-val-01-behavior-contract-standard | #279 | ---`
+  - Blocked by: none (matches proposal declaration of no hard blockers)
+  - Wave 1.5 position: First in wave alongside cli-val-02 (no blockers -- start immediately after Wave 1)
+- **GitHub Issue consistency**: PASS
+  - Issue #279 is OPEN, title "[Change] CLI Behavior Contract Standard"
+  - Issue body matches proposal scope (YAML schema, pilot files, validation tool, documentation)
+  - No "Blocked by" references in issue body (correct -- no blockers)
+  - Labels: enhancement, change-proposal
+- **Cross-artifact dependency consistency**: PASS
+  - Proposal states no hard blockers
+  - CHANGE_ORDER.md shows no blockers for #279
+  - GitHub issue #279 shows no blocker references
+  - All three sources agree
+
+## Breaking-Change Analysis
+
+- **Breaking changes detected:** 0
+- **Interface changes:** None. This change introduces new files only (schema, validation tool, scenario files). No existing production code, interfaces, contracts, or APIs are modified.
+- **Risk assessment:** No breaking-change risk. All artifacts are additive.
+
+## Impact Assessment
+
+- **Impact Level:** Low
+- **Code Impact:** No production CLI code changes. New test infrastructure files only.
+- **Test Impact:** New schema validation tests in `tests/unit/`. New scenario files in `tests/cli-contracts/`. No changes to existing tests.
+- **Documentation Impact:** New contributor page in `docs/` describing CLI behavior contract format.
+- **Release Impact:** Patch/Minor (additive only, no breaking changes)
+
+## Validation Outcome
+
+- Required artifacts are present: `proposal.md`, `design.md`, `specs/cli-behavior-contracts/spec.md`, `tasks.md`.
+- All format checks pass with minor non-blocking observations noted.
+- Dependency declarations are consistent across proposal, CHANGE_ORDER.md, and GitHub issue #279.
+- No breaking changes detected.
+- Change is ready for implementation-phase intake. No prerequisites to satisfy.

openspec/changes/cli-val-01-behavior-contract-standard/design.md

@@ -0,0 +1,73 @@
+## Context
+
+This change establishes the foundational artifact format for CLI end-user validation. It defines how command behavior expectations are recorded, validated, and consumed by downstream validation changes (cli-val-03, cli-val-04, cli-val-06). No production CLI code is modified.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Define a YAML schema for CLI behavior scenarios that is both human-readable and machine-executable
+- Pilot the format on 2-3 command groups to validate usability
+- Provide a schema validation tool that integrates with the existing hatch script system
+- Keep the format simple enough for AI copilots to generate reliably
+
+**Non-Goals:**
+
+- No test runner implementation (that is cli-val-04)
+- No CI integration (that is cli-val-05)
+- No changes to production CLI commands
+- No snapshot testing infrastructure (that is cli-val-02)
+
+## Decisions
+
+- Store scenario files in `tests/cli-contracts/` to keep them adjacent to existing test tiers
+- Use YAML (not JSON or TOML) for scenario files — aligns with existing SpecFact config patterns and is copilot-friendly
+- Separate patterns and anti-patterns within the same file using a `type` field rather than separate files — keeps related behavior together
+- Use JSON Schema for the schema definition — enables validation with `jsonschema` (already a project dependency)
+- Pilot on three command groups: one with many args (e.g., `backlog ceremony`), one with file I/O (e.g., `validate`), one simple (e.g., `--help`/`--version`)
+
+## Schema Design
+
+```yaml
+# tests/cli-contracts/schema/cli-scenario.schema.yaml
+feature: string           # command group name (e.g., "spec validate")
+scenarios:
+  - name: string          # human-readable scenario name
+    type: pattern | anti-pattern
+    argv: [string]        # exact command tokens
+    context:              # optional workspace setup
+      requires: string    # "empty-repo" | "sample-bundle" | "initialized-project"
+      env: {key: value}   # environment variable overrides
+      stdin: string       # stdin input (if any)
+    expect:
+      exit: int           # exact exit code (for patterns)
+      exit_nonzero: bool  # true for anti-patterns
+      stdout_contains: [string]
+      stdout_regex: [string]
+      stderr_contains: [string]
+      stderr_regex: [string]
+      no_traceback: bool  # assert no Python traceback in output
+    fs:
+      creates: [string]   # files expected to be created
+      modifies: [string]  # files expected to be modified
+      forbidden: [string] # files that must NOT be created/modified
+```
+
+## Risks / Trade-offs
+
+- [Schema rigidity] -> Mitigation: start with a minimal schema and extend per downstream change needs; version the schema
+- [YAML complexity for many scenarios] -> Mitigation: one file per command group keeps files manageable; add `$ref` support later if needed
+- [Scenario maintenance burden] -> Mitigation: cli-val-06 provides copilot generation; cli-val-05 CI catches stale scenarios
+
+## Migration Plan
+
+1. Define schema and validation tool
+2. Create pilot scenario files for 3 command groups
+3. Validate all pilots pass schema validation
+4. Document format in `docs/` for contributors
+5. Downstream changes (cli-val-03/04/06) consume the format
+
+## Open Questions
+
+- Exact command groups for the pilot: recommend `backlog ceremony standup`, `validate`, and root `specfact --help`/`--version`
+- Whether to support YAML anchors and aliases for reducing repetition in large scenario files

openspec/changes/cli-val-01-behavior-contract-standard/proposal.md

@@ -0,0 +1,39 @@
+# Change: CLI Behavior Contract Standard
+
+## Why
+
+Every SpecFact CLI feature should have a single artifact that answers: "What does this command do, and what happens when you use it wrong?" Today this knowledge lives scattered across 73+ test files, docstrings, and docs pages. When a user reports unexpected behavior, there is no single source of truth to check the expected CLI behavior against. A machine-readable behavior contract per command group — recording exact argv, expected exit codes, output patterns, and filesystem side effects — closes this gap and becomes the foundation for all downstream validation (snapshot tests, acceptance runners, anti-pattern suites, CI gates).
+
+## What Changes
+
+- **NEW**: YAML schema definition for CLI behavior scenarios (`tests/cli-contracts/schema/cli-scenario.schema.yaml`) — defines the contract format for command groups
+- **NEW**: Scenario files for pilot command groups stored in `tests/cli-contracts/` — one YAML file per command group recording patterns (happy paths) and anti-patterns (misuse/invalid input)
+- **NEW**: Schema validation utility (`tools/validate_cli_contracts.py`) that validates scenario YAML files against the schema
+- **EXTEND**: `openspec/config.yaml` context to reference CLI behavior contracts as a recognized artifact type
+- **EXTEND**: Documentation in `docs/` to describe the CLI behavior contract format and authoring guidelines
+
+## Capabilities
+
+### New Capabilities
+
+- `cli-behavior-contracts`: A YAML-based schema and authoring standard for declaring CLI command behavior expectations — exact argv, required context, expected exit class (success/failure), stdout/stderr patterns, and filesystem diff expectations — separated into patterns (happy paths) and anti-patterns (misuse).
+
+## Impact
+
+- **Affected specs**: No existing specs modified; new spec delta defines the schema and authoring process
+- **Affected code**: New schema file and validation tool only; no production CLI code changes
+- **Integration points**: Consumed by cli-val-03 (anti-patterns), cli-val-04 (acceptance runner), cli-val-06 (copilot generation)
+- **Documentation impact**: New page in `docs/` describing the CLI behavior contract format for contributors
+
+## Dependencies
+
+- No hard blockers — this is the foundation change with no prerequisites
+- Downstream dependents: cli-val-03-misuse-safety-proof, cli-val-04-acceptance-test-runner, cli-val-06-copilot-test-generation
+
+## Source Tracking
+
+<!-- source_repo: nold-ai/specfact-cli -->
+- **GitHub Issue**: #279
+- **Issue URL**: https://github.com/nold-ai/specfact-cli/issues/279
+- **Repository**: nold-ai/specfact-cli
+- **Last Synced Status**: proposed

openspec/changes/cli-val-01-behavior-contract-standard/specs/cli-behavior-contracts/spec.md

@@ -0,0 +1,63 @@
+## ADDED Requirements
+
+### Requirement: CLI Behavior Contract Schema
+
+The system SHALL provide a YAML schema that defines the structure for CLI behavior scenario files, enabling machine-readable declaration of expected command behavior.
+
+#### Scenario: Schema validates a well-formed scenario file
+
+- **GIVEN** a scenario YAML file with feature name, argv list, expect block (exit code, stdout/stderr patterns), and fs block (creates/modifies)
+- **WHEN** the schema validator runs against the file
+- **THEN** validation passes with no errors
+- **AND** all required fields are present and correctly typed.
+
+#### Scenario: Schema rejects a scenario file missing required fields
+
+- **GIVEN** a scenario YAML file missing the `argv` field in a scenario entry
+- **WHEN** the schema validator runs against the file
+- **THEN** validation fails with a descriptive error identifying the missing field
+- **AND** the error message includes the scenario name and line context.
+
+#### Scenario: Schema supports both pattern and anti-pattern categorization
+
+- **GIVEN** a scenario YAML file with entries categorized as `type: pattern` and `type: anti-pattern`
+- **WHEN** the file is parsed
+- **THEN** patterns and anti-patterns are distinguishable programmatically
+- **AND** anti-patterns require `exit_nonzero: true` in the expect block.
+
+### Requirement: Pilot Scenario Files
+
+The system SHALL include pilot scenario files for at least three command groups demonstrating the contract format across different command characteristics.
+
+#### Scenario: Pilot covers a command with many arguments
+
+- **GIVEN** a scenario file for a command group with multiple required and optional arguments
+- **WHEN** the file includes happy-path and misuse scenarios
+- **THEN** at least 3 pattern scenarios and 3 anti-pattern scenarios are defined
+- **AND** each scenario records exact argv, expected exit code, and output patterns.
+
+#### Scenario: Pilot covers a command with file I/O
+
+- **GIVEN** a scenario file for a command that reads or writes files
+- **WHEN** the file includes filesystem expectation blocks
+- **THEN** the `fs.creates` and `fs.modifies` fields list expected file paths
+- **AND** anti-patterns assert `fs.creates: []` and `fs.modifies: []` (no side effects on failure).
+
+#### Scenario: Pilot covers a simple informational command
+
+- **GIVEN** a scenario file for a command like `--help` or `--version`
+- **WHEN** the file defines expected output
+- **THEN** stdout patterns match documented help text structure
+- **AND** exit code is 0 for valid invocation.
+
+### Requirement: Schema Validation Tool
+
+The system SHALL provide a validation tool that checks scenario YAML files against the schema and reports errors.
+
+#### Scenario: Validation tool runs via hatch script
+
+- **GIVEN** scenario files exist in `tests/cli-contracts/`
+- **WHEN** `hatch run validate-cli-contracts` is executed
+- **THEN** all scenario files are validated against the schema
+- **AND** errors are reported with file path and line context
+- **AND** exit code is 0 when all files pass, non-zero when any fail.