Skip to content

V&V coverage matrix dashboard via rivet coverage (per-commit, per-repo, per-technique) #188

@avrabe

Description

@avrabe

Part of the V&V coverage initiative.

Problem

Today the V&V coverage matrix across pulseengine (which repo runs Verus / Kani / Rocq / Lean / proptest / tokio-loom / fuzz / Miri / sanitizers / mutation / bench / etc.) lives only in ad-hoc documentation and human-read matrices. We need a machine-readable, continuously-updated dashboard so:

  • New repos auto-appear with blanks to fill in
  • Adoption progress is visible to assessors without reading GitHub issues
  • Per-commit drift (a technique silently disabled) triggers an alarm

Acceptance

  • Extend rivet schema to capture technique-adoption status per repo:
    • techniques_applied: [verus, kani, rocq, lean, proptest, loom, miri, asan, fuzz, mutation, criterion, ...]
    • techniques_gated_in_ci: [...] (subset that's enforced, not just present)
  • New subcommand rivet coverage --matrix emits a rendered matrix (markdown, HTML, JSON)
  • Aggregator that walks the pulseengine org and composes a cross-repo matrix (CI-friendly)
  • Published dashboard (GitHub Pages? rivet-web component? Something deterministic from the YAML)
  • Integration: pulseengine.eu surfaces the live matrix
  • Per-technique trend line (adoption over time) — stretch goal
  • Per-standard mapping view: "which DO-178C objectives are backed by proofs in which repo?" — stretch goal

Notes

  • This operationalizes the "V&V coverage initiative" permanently — the tracker becomes a live system, not a one-off sweep
  • Closes the gap between "we have lots of verification" and "we can show assessors which verification applies to which requirement"
  • Related: the cross-domain standards mapping (ISO 26262, IEC 61508, EN 50128, IEC 62304, DO-178C, ECSS-Q-ST-80C) can drive the per-standard view

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions