feat: add thorough codebase validation (validation-01, #163)

CHANGELOG.md

@@ -8,17 +8,24 @@ All notable changes to this project will be documented in this file.
 **Important:** Changes need to be documented below this block as this is the header section. Each section should be separated by a horizontal rule. Newer changelog entries need to be added on top of prior ones to keep the history chronological with most recent changes first.
 
 ---
-
 ## [0.34.0] - 2026-02-18
 
 ### Added
 
+- **Thorough codebase validation** (validation-01, [#163](https://github.com/nold-ai/specfact-cli/issues/163))
+  - `specfact repro --crosshair-per-path-timeout N` to run CrossHair with a higher per-path timeout (deep validation).
+  - Reference doc [Thorough Codebase Validation](docs/reference/thorough-codebase-validation.md) covering quick check (`specfact repro`), thorough contract-decorated (`hatch run contract-test-full`), sidecar for unmodified code, and dogfooding (repro + contract-test-full on specfact-cli).
+  - Unit test and TDD evidence for CrossHair per-path timeout passthrough.
 - **Init module discovery alignment** (backlog-core-01): `specfact init` now uses the same module discovery roots as command registration (`discover_all_package_metadata()`), so `--list-modules`, `--enable-module`, and `--disable-module` operate on all discovered modules including workspace-level ones (e.g. `modules/backlog-core/`). Closes [#116](https://github.com/nold-ai/specfact-cli/issues/116) scope for init-module-discovery-alignment.
 
 ### Changed
 
 - `specfact init` module state and validation now build from `discover_all_package_metadata()` instead of `discover_package_metadata(get_modules_root())`, aligning enable/disable and list-modules with runtime command discovery.
 
+### Fixed
+
+- `specfact repro --crosshair-per-path-timeout 0` (or negative) now fails with a clear error instead of being silently ignored; CLI rejects non-positive CrossHair per-path timeout values.
+
 ---
 
 ## [0.33.0] - 2026-02-17

docs/_layouts/default.html

@@ -171,6 +171,7 @@ <h2 class="docs-sidebar-title">
               <ul>
                 <li><a href="{{ '/reference/' | relative_url }}">Reference Documentation</a></li>
                 <li><a href="{{ '/reference/commands/' | relative_url }}">Command Reference</a></li>
+                <li><a href="{{ '/reference/thorough-codebase-validation/' | relative_url }}">Thorough Codebase Validation</a></li>
                 <li><a href="{{ '/reference/authentication/' | relative_url }}">Authentication</a></li>
                 <li><a href="{{ '/architecture/' | relative_url }}">Architecture</a></li>
                 <li><a href="{{ '/modes/' | relative_url }}">Operational Modes</a></li>

docs/index.md

@@ -99,6 +99,7 @@ Why this matters:
 - **[Extending ProjectBundle](guides/extending-projectbundle.md)** - Add namespaced custom fields to Feature/ProjectBundle (arch-07)
 - **[Using Module Security and Extensions](guides/using-module-security-and-extensions.md)** - Use arch-06 (module security) and arch-07 (schema extensions) from CLI and as a module author
 - **[Sidecar Validation](guides/sidecar-validation.md)** 🆕 - Validate external codebases without modifying source
+- **[Thorough Codebase Validation](reference/thorough-codebase-validation.md)** - Quick check, contract-full, sidecar, dogfooding
 - **[UX Features](guides/ux-features.md)** - Progressive disclosure, context detection, intelligent suggestions
 - **[Use Cases](guides/use-cases.md)** - Real-world scenarios and workflows
 - **[IDE Integration](guides/ide-integration.md)** - Set up slash commands in your IDE

docs/reference/README.md

@@ -11,6 +11,7 @@ Complete technical reference for SpecFact CLI.
 ## Available References
 
 - **[Commands](commands.md)** - Complete command reference with all options
+- **[Thorough Codebase Validation](thorough-codebase-validation.md)** - Quick check, contract-decorated, sidecar, and dogfooding
 - **[Command Syntax Policy](command-syntax-policy.md)** - Source-of-truth argument syntax conventions for docs
 - **[Authentication](authentication.md)** - Device code auth flows and token storage
 - **[Architecture](architecture.md)** - Technical design, module structure, and internals

docs/reference/thorough-codebase-validation.md

@@ -0,0 +1,104 @@
+---
+layout: default
+title: Thorough Codebase Validation
+permalink: /reference/thorough-codebase-validation/
+description: How to run in-depth validation (quick check, contract-decorated, sidecar, dogfooding).
+---
+
+# Thorough Codebase Validation
+
+This reference describes how to run thorough in-depth validation in different modes: quick check, contract-decorated codebases, sidecar for unmodified code, and dogfooding SpecFact CLI on itself.
+
+## Validation Modes
+
+| Mode | When to use | Primary command(s) |
+|------|-------------|---------------------|
+| **Quick check** | Fast local/CI gate (lint, type-check, CrossHair with default budget) | `specfact repro --repo <path>` |
+| **Thorough (contract-decorated)** | Repo already uses `@icontract` / `@beartype`; run full contract stack | `hatch run contract-test-full` |
+| **Sidecar (unmodified code)** | Third-party or legacy repo; no edits to target source | `specfact repro --repo <path> --sidecar --sidecar-bundle <bundle>` |
+| **Dogfooding** | Validate the specfact-cli repo with the same pipeline | `specfact repro --repo .` + `hatch run contract-test-full` (optional sidecar) |
+
+## 1. Quick check (`specfact repro`)
+
+Run the standard reproducibility suite (ruff, semgrep if config exists, basedpyright, CrossHair, optional pytest contracts/smoke):
+
+```bash
+specfact repro --repo .
+specfact repro --repo /path/to/external/repo --verbose
+```
+
+- **Time budget**: Default 120s; use `--budget N` (advanced) to change.
+- **Deep CrossHair**: To increase per-path timeout for CrossHair (e.g. for critical modules), use `--crosshair-per-path-timeout N` (seconds; N must be positive). Default behavior is unchanged when not set.
+
+```bash
+specfact repro --repo . --crosshair-per-path-timeout 60
+```
+
+Required env: none. Optional: `[tool.crosshair]` in `pyproject.toml` (e.g. from `specfact repro setup`).
+
+## 2. Thorough validation for contract-decorated codebases
+
+When your repo already has `@icontract` and `@beartype` on public APIs, use the full contract-test stack:
+
+```bash
+hatch run contract-test-full
+```
+
+This runs:
+
+- Runtime contract validation (`contract-test-contracts`)
+- CrossHair exploration (`contract-test-exploration`)
+- Scenario tests with contract references (`contract-test-scenarios`)
+
+Exploration timeout can be configured via `[tool.crosshair]` or env (e.g. `STANDARD_CROSSHAIR_TIMEOUT`). For deeper CrossHair analysis on critical paths, run CrossHair directly with a higher per-path timeout:
+
+```bash
+crosshair check --per_path_timeout=60 src/your_critical_module/
+```
+
+Document this as the recommended thorough path for contract-decorated code; CI can invoke `hatch run contract-test-full` for PR validation.
+
+## 3. Sidecar validation (unmodified code)
+
+For repositories you cannot or do not want to modify (no contract decorators added):
+
+```bash
+specfact repro --repo <path> --sidecar --sidecar-bundle <bundle-name>
+```
+
+- Main repro checks run first (lint, semgrep, type-check, CrossHair if available).
+- Then sidecar validation runs: unannotated detection, harness generation, CrossHair/Specmatic on generated harnesses. No files in the target repo are modified.
+- If CrossHair is not installed or the bundle is invalid, sidecar is skipped or partial with clear messaging; non-zero exit only for main check failures (sidecar can be advisory).
+
+See [Sidecar Validation Guide](/guides/sidecar-validation/) for setup and bundle configuration.
+
+## 4. Dogfooding (SpecFact CLI on itself)
+
+Maintainers can validate the specfact-cli repository with the same pipeline:
+
+1. **Repro + contract-test-full** (recommended minimum):
+
+   ```bash
+   specfact repro --repo .
+   hatch run contract-test-full
+   ```
+
+2. **Optional sidecar** (to cover unannotated code in specfact-cli):
+
+   ```bash
+   specfact repro --repo . --sidecar --sidecar-bundle <bundle-name>
+   ```
+
+Use the same commands in a CI job or release checklist so specfact-cli validates itself before release. No repo-specific code is required beyond existing repro and contract-test tooling.
+
+## Copy-paste summary
+
+| Goal | Commands |
+|------|----------|
+| Quick gate | `specfact repro --repo .` |
+| Deep CrossHair (repro) | `specfact repro --repo . --crosshair-per-path-timeout 60` |
+| Full contract stack | `hatch run contract-test-full` |
+| Unmodified repo | `specfact repro --repo <path> --sidecar --sidecar-bundle <name>` |
+| Dogfooding | `specfact repro --repo .` then `hatch run contract-test-full`; optionally add `--sidecar --sidecar-bundle <name>` to repro |
+
+Required env/config: optional `[tool.crosshair]` in `pyproject.toml`; for sidecar, a valid sidecar bundle and CrossHair installed when sidecar CrossHair is used.

openspec/changes/validation-01-deep-validation/TDD_EVIDENCE.md

@@ -0,0 +1,23 @@
+# TDD Evidence: validation-01-deep-validation
+
+## Behavior change: CrossHair per-path timeout option
+
+### Pre-implementation (failing test)
+
+- **Test**: `tests/unit/validators/test_repro_checker.py::TestReproChecker::test_repro_checker_crosshair_per_path_timeout_passed_to_command`
+- **Command**: `hatch test -- tests/unit/validators/test_repro_checker.py -v -k "crosshair_per_path"`
+- **Timestamp**: 2026-02-18 (before implementation)
+- **Result**: Failed — `ReproChecker` had no `crosshair_per_path_timeout` and CrossHair command did not include `--per_path_timeout`.
+
+### Post-implementation (passing test)
+
+- **Command**: `hatch test -- tests/unit/validators/test_repro_checker.py -v -k "crosshair_per_path"`
+- **Timestamp**: 2026-02-18
+- **Result**: Passed — `ReproChecker(repo_path=..., crosshair_per_path_timeout=60)` produces a CrossHair invocation with `--per_path_timeout` and `60` in the command list.
+
+### Implementation summary
+
+1. Added `crosshair_per_path_timeout: int | None = None` to `ReproChecker.__init__` and stored on `self`.
+2. In `run_all_checks()`, when building `crosshair_base`, append `--per_path_timeout` and the value when `self.crosshair_per_path_timeout` is set and > 0.
+3. Added `--crosshair-per-path-timeout` option to repro command in `src/specfact_cli/modules/repro/src/commands.py` and passed through to `ReproChecker`.
+4. Unit test mocks `subprocess.run` at `specfact_cli.validators.repro_checker.subprocess.run`, runs `run_all_checks()` with `crosshair_per_path_timeout=60`, and asserts the CrossHair call includes `--per_path_timeout` and `60`.

openspec/changes/validation-01-deep-validation/tasks.md

@@ -2,46 +2,46 @@
 
 ## 1. Create git worktree branch from dev
 
-- [ ] 1.1 Ensure primary checkout is on dev and up to date: `git checkout dev && git pull origin dev`
-- [ ] 1.2 Create worktree branch: `scripts/worktree.sh create feature/add-thorough-codebase-validation`; if issue exists, link it with `gh issue develop <issue-number> --repo nold-ai/specfact-cli --name feature/add-thorough-codebase-validation`
-- [ ] 1.3 Verify branch in worktree: `git worktree list` includes the branch path; then run `git branch --show-current` inside that worktree.
+- [x] 1.1 Ensure primary checkout is on dev and up to date: `git checkout dev && git pull origin dev`
+- [x] 1.2 Create worktree branch: `scripts/worktree.sh create feature/validation-01-deep-validation` (used branch name aligned with CHANGE_ORDER).
+- [x] 1.3 Verify branch in worktree: `git worktree list` includes the branch path; then run `git branch --show-current` inside that worktree.
 
 ## 2. Verify spec deltas (SDD: specs first)
 
-- [ ] 2.1 Confirm `specs/codebase-validation-depth/spec.md` exists and is complete (ADDED requirements, Given/When/Then scenarios).
-- [ ] 2.2 Map scenarios to implementation: Sidecar unmodified, Sidecar optional when CrossHair missing, Full contract-stack, CrossHair deep, Dogfooding commands, Dogfooding optional sidecar, Documentation of validation modes.
+- [x] 2.1 Confirm `specs/codebase-validation-depth/spec.md` exists and is complete (ADDED requirements, Given/When/Then scenarios).
+- [x] 2.2 Map scenarios to implementation: Sidecar unmodified, Sidecar optional when CrossHair missing, Full contract-stack, CrossHair deep, Dogfooding commands, Dogfooding optional sidecar, Documentation of validation modes.
 
 ## 3. Optional: Deep CrossHair / repro options
 
-- [ ] 3.1 In `src/specfact_cli/commands/repro.py`: add optional `--crosshair-per-path-timeout N` (default: use existing budget behavior) so users can increase CrossHair depth for repro runs.
-- [ ] 3.2 In `src/specfact_cli/validators/repro_checker.py`: when building CrossHair command, append `--per_path_timeout N` when repro option is set; keep default unchanged.
-- [ ] 3.3 Add unit or integration test that repro with `--crosshair-per-path-timeout` passes through to CrossHair command (or skip if deferred to docs-only).
-- [ ] 3.4 Run format and type-check: `hatch run format`, `hatch run type-check`.
+- [x] 3.1 In repro command (implementation in `src/specfact_cli/modules/repro/src/commands.py`): add optional `--crosshair-per-path-timeout N` (default: use existing budget behavior).
+- [x] 3.2 In `src/specfact_cli/validators/repro_checker.py`: when building CrossHair command, append `--per_path_timeout N` when option is set; keep default unchanged.
+- [x] 3.3 Add unit test that repro with `--crosshair-per-path-timeout` passes through to CrossHair command: `test_repro_checker_crosshair_per_path_timeout_passed_to_command`.
+- [x] 3.4 Run format and type-check: `hatch run format`, `hatch run type-check`.
 
 ## 4. Documentation: Thorough codebase validation
 
-- [ ] 4.1 Add or extend a reference section "Thorough codebase validation" (e.g. in `docs/reference/` or under existing validation doc) covering: (1) quick check (`specfact repro`), (2) thorough contract-decorated (`hatch run contract-test-full`), (3) sidecar for unmodified code (`specfact repro --sidecar --sidecar-bundle <bundle>`), (4) dogfooding (repro + contract-test-full on specfact-cli; optional sidecar).
-- [ ] 4.2 Document optional deep CrossHair: how to run CrossHair with higher per-path timeout (repro flag or `crosshair check --per_path_timeout=60 <module>`); optional module list for critical paths.
-- [ ] 4.3 Add dogfooding checklist or CI note: exact commands and order for validating specfact-cli (repro + contract-test-full; optional sidecar); link from README or contributing guide if appropriate.
-- [ ] 4.4 Ensure docs are copy-pasteable; state any required env or config (e.g. `[tool.crosshair]`, sidecar bundle).
-- [ ] 4.5 If adding a new doc page: set front-matter (layout, title, permalink, description) and update `docs/_layouts/default.html` sidebar if needed.
+- [x] 4.1 Add or extend a reference section "Thorough codebase validation" in `docs/reference/thorough-codebase-validation.md` covering: (1) quick check, (2) thorough contract-decorated, (3) sidecar, (4) dogfooding.
+- [x] 4.2 Document optional deep CrossHair: repro flag `--crosshair-per-path-timeout N` and `crosshair check --per_path_timeout=60 <module>`.
+- [x] 4.3 Add dogfooding checklist in same doc: exact commands and order (repro + contract-test-full; optional sidecar).
+- [x] 4.4 Docs are copy-pasteable; required env/config stated (`[tool.crosshair]`, sidecar bundle).
+- [x] 4.5 New doc page has front-matter; `docs/_layouts/default.html` and `docs/reference/README.md` updated.
 
 ## 5. Optional: CI job for thorough validation (dogfooding)
 
-- [ ] 5.1 Add or update a CI job (e.g. in `.github/workflows/`) that runs `specfact repro --repo .` and `hatch run contract-test-full` (or equivalent) so specfact-cli validates itself on PR or nightly. Use reasonable timeouts to avoid flakiness.
-- [ ] 5.2 Document the job in the "Thorough codebase validation" section; mark as optional if job is added in a follow-up.
+- [ ] 5.1 Add or update a CI job (deferred to follow-up; documented commands suffice for this change).
+- [x] 5.2 Document the commands in "Thorough codebase validation"; CI job marked optional/follow-up.
 
 ## 6. Quality gates
 
-- [ ] 6.1 Run format and type-check: `hatch run format`, `hatch run type-check`.
-- [ ] 6.2 Run contract test: `hatch run contract-test`.
-- [ ] 6.3 Run full test suite: `hatch run smart-test-full` (or `hatch test --cover -v`).
-- [ ] 6.4 Ensure any new or modified public APIs have `@icontract` and `@beartype` where applicable.
+- [x] 6.1 Run format and type-check: `hatch run format`, `hatch run type-check`.
+- [x] 6.2 Run contract test: `hatch run contract-test`.
+- [x] 6.3 Run full test suite: `hatch run smart-test-full` (or `hatch test --cover -v`); validator unit tests passed.
+- [x] 6.4 New/modified public APIs: `ReproChecker.__init__` and repro CLI already use contracts/beartype; no new decorators required.
 
 ## 7. Documentation research and review (per openspec/config.yaml)
 
-- [ ] 7.1 Identify affected documentation: new or extended "Thorough codebase validation" section; README or contributing link if added; no new top-level pages unless created in task 4.
-- [ ] 7.2 Verify front-matter and sidebar if a new page was added; confirm no broken links.
+- [x] 7.1 Affected documentation: new `docs/reference/thorough-codebase-validation.md`; reference README and sidebar updated.
+- [x] 7.2 Front-matter and sidebar updated; no broken links.
 
 ## 8. Create Pull Request to dev
 

src/specfact_cli/modules/repro/src/commands.py

@@ -173,6 +173,11 @@ def main(
         "--crosshair-required",
         help="Fail if CrossHair analysis is skipped/failed (strict contract exploration mode)",
     ),
+    crosshair_per_path_timeout: int | None = typer.Option(
+        None,
+        "--crosshair-per-path-timeout",
+        help="CrossHair per-path timeout in seconds (deep validation; default: use existing budget behavior)",
+    ),
     # Advanced/Configuration
     budget: int = typer.Option(
         120,
@@ -233,6 +238,8 @@ def main(
         raise typer.BadParameter("Repo path must exist and be directory")
     if budget <= 0:
         raise typer.BadParameter("Budget must be positive")
+    if crosshair_per_path_timeout is not None and crosshair_per_path_timeout <= 0:
+        raise typer.BadParameter("CrossHair per-path timeout must be positive")
     if not _is_valid_output_path(out):
         raise typer.BadParameter("Output path must exist if provided")
     if sidecar and not sidecar_bundle:
@@ -249,6 +256,8 @@ def main(
         console.print("[dim]Auto-fix: enabled[/dim]")
     if crosshair_required:
         console.print("[dim]CrossHair required: enabled[/dim]")
+    if crosshair_per_path_timeout is not None:
+        console.print(f"[dim]CrossHair per-path timeout: {crosshair_per_path_timeout}s[/dim]")
     console.print()
 
     # Ensure structure exists
@@ -269,6 +278,7 @@ def main(
             fail_fast=fail_fast,
             fix=fix,
             crosshair_required=crosshair_required,
+            crosshair_per_path_timeout=crosshair_per_path_timeout,
         )
 
         # Detect and display environment manager before starting progress spinner