docs: AI-assisted safety + cyber engineering — HITL contract

[avrabe]

Summary

Single-commit design doc docs/design/ai-safety-cyber-hitl.md (~2500 words) that frames the recurring objection "but a qualified human still has to sign off" as the shape of AI assistance in a regulated SDLC, not an argument against it.

Chosen one-sentence frame: AI proposes structure; a qualified human owns judgment; every transition between the two is a separately-stamped, git-reviewable event.

Contents

• §1 TL;DR — one-sentence frame, with "everything follows from this" framing.
• §2 The regulatory reality — 12-row table across ISO 26262:2018, IEC 61508-1, IEC 62304, DO-178C / DO-330, EN 50128, ISO/SAE 21434, ISO 27001, IEC 62443-4-1, ASPICE 4.0, EU AI Act Art. 14, NIST AI RMF. Each row: role, what they sign, what AI can NEVER do. Clause numbers flagged (unverified clause-level) where the primary text could not be fetched this session.
• §3 How existing tools handle the tension — Jama Advisor, Polarion, Codebeamer, Ansys medini, BTC, TÜV. Distinguishes honest framing (AI output labelled, sign-off a separate state, AI doesn't sign) from overclaiming.
• §4 The pattern — 6-row table mapping authoring/linking/validation/gap-detection/summarisation/sign-off onto AI role, human role, and concrete rivet commands.
• §5 Rivet's four-point HITL contract — what's in main today vs. v0.5.0 proposals:
  1. Every AI-authored artifact carries provenance (today — schemas/common.yaml ai-generated-needs-review rule; proposed: promote to error on status: approved and add self-approval-loophole lint).
  2. Human sign-off as a separate stamp (today — rivet stamp --reviewed-by; gaps: structured rationale, rivet approve alias, Part 11 e-signature).
  3. rivet audit-trail <id> (v0.5.0 proposal — chronological view over git history + provenance transitions).
  4. Structural-only validator boundary (today — rivet validate never assesses credibility).
• §6 FAQ — pocket answers to five customer objections (can't-do-safety, hallucination, liability, Copilot-difference, TÜV-acceptance).
• §7 What rivet explicitly does NOT claim — six-item anti-overclaim list.
• §8 Cross-references + implementation backlog — proposed text updates to docs/what-is-rivet.md (not made here); five-item v0.5.0 backlog.

Honest constraints

• Live WebFetch / WebSearch were unavailable this session; every external regulatory clause citation and vendor marketing phrase is flagged *(unverified)* per the task constraint "mark unverified."
• Rivet does NOT have rivet approve or rivet audit-trail subcommands today. Both are called out as v0.5.0 proposals, not claimed as features.
• Word count: 2498 (under the 2500 limit).

Refs: FEAT-001 (Evidence-as-Code positioning), REQ-002 (STPA artifact support — the cyber-safety joint analysis pattern), REQ-030 (formal verification — the structural-only enforcement boundary).

Test plan

• Product lead reviews §1 frame sentence for use in sales materials.
• Safety lead verifies clause numbers in §2 against paid copies of ISO 26262, IEC 61508, IEC 62304, DO-178C, EN 50128, ISO/SAE 21434 before external quotation.
• Competitive research re-fetches §3 vendor marketing phrases (Jama, Polarion, Codebeamer, Ansys, BTC, TÜV) before external use.
• PM evaluates §5/§8 backlog items (promote ai-generated-needs-review to error on approved, self-approval-loophole lint, rivet approve alias, rivet audit-trail <id>, structured reviewed-by.rationale for ASIL ≥ B / CAL ≥ 2) for v0.5.0 scope.

🤖 Generated with Claude Code