Skip to content

feat: add thorough codebase validation (validation-01, #163)#272

Merged
djm81 merged 4 commits intodevfrom
feature/validation-01-deep-validation
Feb 18, 2026
Merged

feat: add thorough codebase validation (validation-01, #163)#272
djm81 merged 4 commits intodevfrom
feature/validation-01-deep-validation

Conversation

@djm81
Copy link
Copy Markdown
Collaborator

@djm81 djm81 commented Feb 18, 2026

Description

Implements validation-01-deep-validation (OpenSpec): thorough codebase validation with optional deep CrossHair and documented modes (quick check, contract-decorated, sidecar, dogfooding).

  • Repro: Add --crosshair-per-path-timeout N to specfact repro; when set, CrossHair is invoked with --per_path_timeout N for deeper exploration.
  • ReproChecker: New optional crosshair_per_path_timeout parameter; when building the CrossHair command, appends --per_path_timeout N when set (default behaviour unchanged).
  • Docs: New reference page Thorough codebase validation (docs/reference/thorough-codebase-validation.md) covering: quick check (specfact repro), thorough contract-decorated (hatch run contract-test-full), sidecar for unmodified code, dogfooding (repro + contract-test-full on specfact-cli); copy-paste commands and optional deep CrossHair.
  • Tests: Unit test test_repro_checker_crosshair_per_path_timeout_passed_to_command; TDD evidence in openspec/changes/validation-01-deep-validation/TDD_EVIDENCE.md.

New Features #163

Contract References: ReproChecker.__init__ (existing @require/@ensure/@beartype; new parameter crosshair_per_path_timeout); repro command in modules/repro (typer options). No new contract decorators added; existing contracts cover the new code path.

Type of Change

Please check all that apply:

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔒 Contract enforcement (adding/updating @icontract decorators)
  • 🧪 Test enhancement (scenario tests, property-based tests)
  • 🔧 Refactoring (code improvement without functionality change)

Contract-First Testing Evidence

Required for all changes affecting CLI commands or public APIs:

Contract Validation

  • Runtime contracts added/updated (existing @icontract on ReproChecker; new parameter covered by existing contracts)
  • Type checking enforced (@beartype on ReproChecker and repro command)
  • CrossHair exploration completed: hatch run contract-test-exploration
  • Contract violations reviewed and addressed (none introduced)

Test Execution

  • Contract validation: hatch run contract-test-contracts
  • Contract exploration: hatch run contract-test-exploration
  • Scenario tests: hatch run contract-test-scenarios
  • Full test suite: hatch run contract-test-full

Test Quality

  • CLI commands tested with typer test client (repro command exercised via unit test on ReproChecker)
  • Edge cases covered with Hypothesis property tests
  • Error handling tested with invalid inputs (existing tests; new path covered by test_repro_checker_crosshair_per_path_timeout_passed_to_command)
  • Rich console output verified manually or with snapshots

How Has This Been Tested?

Contract-First Approach: New behaviour (CrossHair command including --per_path_timeout) is validated by unit test that mocks subprocess.run, runs ReproChecker.run_all_checks() with crosshair_per_path_timeout=60, and asserts the CrossHair invocation includes --per_path_timeout and 60. Existing contracts and beartype cover the new parameter.

Manual Testing

  • Tested CLI commands manually (specfact repro --help shows --crosshair-per-path-timeout)
  • Verified rich console output (dim line when option set)
  • Tested with different input scenarios (with/without option)
  • Checked error messages for clarity

Automated Testing

  • Contract validation passes
  • Property-based tests cover edge cases
  • Scenario tests cover user workflows
  • All existing tests still pass (23/23 in test_repro_checker.py)

Test Environment

  • Python version: 3.11
  • OS: Linux

Checklist

  • My code follows the style guidelines (PEP 8, ruff format, isort)
  • I have performed a self-review of my code
  • I have added/updated contracts (@icontract, @beartype) — existing contracts apply; no new decorators
  • I have added/updated docstrings (Google style)
  • I have made corresponding changes to documentation
  • My changes generate no new warnings (basedpyright, ruff, pylint)
  • All tests pass locally
  • I have added tests that prove my fix/feature works
  • Any dependent changes have been merged

Quality Gates Status

  • Type checking ✅ (hatch run type-check)
  • Linting ✅ (hatch run lint)
  • Contract validation ✅ (hatch run contract-test-contracts)
  • Contract exploration ✅ (hatch run contract-test-exploration)
  • Scenario tests ✅ (hatch run contract-test-scenarios)

Screenshots/Recordings (if applicable)

N/A — CLI help and doc page suffice.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

architecture Architecture and design changes enhancement New feature or request openspec

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Change] Add thorough in-depth codebase validation for sidecar, contract-decorated codebases, and dogfooding

1 participant