Skip to content

feat(ci): frontmatter schema validator — hard-block from day one#196

Merged
klappy merged 4 commits into
mainfrom
feat/frontmatter-schema-validator
May 11, 2026
Merged

feat(ci): frontmatter schema validator — hard-block from day one#196
klappy merged 4 commits into
mainfrom
feat/frontmatter-schema-validator

Conversation

@klappy
Copy link
Copy Markdown
Owner

@klappy klappy commented May 10, 2026

What

Closes the implementation gap in klappy://canon/constraints/frontmatter-validation-before-merge.

Canon already mandates automated frontmatter validation on every writings/ PR — "No Exceptions". The existing canon-quality.yml workflow only ran oddkit_audit (reference-integrity), so frontmatter violations passed CI silently and the renderer silently dropped them from the homepage. That gap caused the May 10 incident (PR #194).

This adds a parallel CI job — frontmatter — that runs scripts/validate-frontmatter.py against writings/ on every PR and push. The validator mirrors the enum and required-field rules in canon/meta/frontmatter-schema.md.

Six rule_ids, each tied to a Known Crash Pattern

rule_id Catches
frontmatter-missing-block No --- delimiters at all
frontmatter-parse-error Malformed YAML
frontmatter-missing-required Universal field absent, OR essay discovery field (type/slug/hook/description) absent on public writings
frontmatter-invalid-enum exposure/voice/tier/audience value outside the canon-allowed set
frontmatter-type-mismatch Quoted boolean (public: "true") or quoted integer (tier: "3")
frontmatter-contradictory public: false + exposure: public

Hard-block from day one

No soft observation cycle. Rationale:

  1. The schema is unambiguous (enums, not subjective judgment).
  2. The renderer's failure mode is silent — no operator signal, just a missing card.
  3. Canon mandates the gate "No Exceptions".
  4. Dry-run against all 40 essays in main returns zero findings. No existing PRs are blocked at flip time.

Why not extend oddkit_audit?

The oddkit_audit spec (docs/oddkit/specs/oddkit-audit.md, DRAFT v2.2) is explicitly KISS — "one check, two rule_ids" — and mandates new checks become "their own thin action — not bolted into this one." Frontmatter validation is also klappy.dev-specific: the enum values belong to one knowledge base, not to oddkit's generic codebase. Per Vodka Architecture, klappy.dev-specific rules belong here. oddkit_audit runs in parallel and is unaffected.

Tests

  • scripts/tests/test_validator.py exercises all five rule classes against fixtures plus the live writings/ directory.
  • Fixtures isolated under scripts/tests/fixtures/, outside the workflow's writings/ scan path.

Local verification:

$ python3 scripts/validate-frontmatter.py
✅ Frontmatter OK — 40 file(s) scanned, 0 findings.

$ python3 scripts/tests/test_validator.py
All validator smoke tests passed.

Canon

  • klappy://canon/constraints/frontmatter-validation-before-mergeAutomation section amended to describe the implementation.
  • klappy://canon/meta/frontmatter-schema — unchanged. Single source of truth for what the validator mirrors.

Note

Medium Risk
Introduces a new hard-blocking CI gate on PRs touching writings/**, which can immediately prevent merges if the validator is wrong or drifts from canon/meta/frontmatter-schema.md. The changes are isolated to CI + a new Python validator, but impact the main development workflow.

Overview
Adds a new hard-blocking frontmatter CI job in .github/workflows/canon-quality.yml that runs scripts/validate-frontmatter.py on writings/, uploads a JSON findings artifact, posts a sticky PR comment/step summary, and fails the workflow if any violations are found.

Introduces scripts/validate-frontmatter.py, a schema-mirroring validator that checks for missing/invalid frontmatter, required fields (including extra discovery fields for public writings), enum violations, type mismatches (quoted booleans/integers), and contradictory public/exposure flags, plus adds smoke tests + fixtures under scripts/tests/.

Updates canon/constraints/frontmatter-validation-before-merge.md to document the new CI implementation, rule IDs, and the “no exceptions” hard-block enforcement.

Reviewed by Cursor Bugbot for commit f1c23b0. Bugbot is set up for automated code reviews on this repo. Configure here.

Closes the implementation gap in klappy://canon/constraints/frontmatter-validation-before-merge.

Canon mandates automated frontmatter validation on every writings/ PR
"No Exceptions". The existing canon-quality.yml workflow only ran
oddkit_audit (which audits klappy:// reference integrity), so frontmatter
violations passed CI silently and the renderer silently dropped them
from the homepage. That gap caused the May 10 incident (PR #194).

This change adds a parallel CI job that runs scripts/validate-frontmatter.py
against writings/ on every PR and push. The validator mirrors the enum and
required-field rules in canon/meta/frontmatter-schema.md.

Five rule_ids, each mapped to a documented Known Crash Pattern:

  - frontmatter-missing-block      — no --- delimiters
  - frontmatter-parse-error        — malformed YAML
  - frontmatter-missing-required   — universal field absent, OR essay
                                     discovery field (type/slug/hook/
                                     description) absent on public writings
  - frontmatter-invalid-enum       — exposure/voice/tier/audience value
                                     outside the canon-allowed set
  - frontmatter-type-mismatch      — quoted boolean or quoted integer
  - frontmatter-contradictory      — public:false + exposure:public

Hard-block from day one. No soft observation cycle. Rationale: the schema
is unambiguous (enums, not subjective judgment), the renderer's failure
mode is silent (no operator signal), canon mandates it 'No Exceptions',
and dry-run against all 40 essays in main returns zero findings — no
existing PRs would be blocked at flip time.

Why not extend oddkit_audit?

  The oddkit_audit spec (docs/oddkit/specs/oddkit-audit.md, DRAFT v2.2)
  is explicitly KISS — 'one check, two rule_ids' — and mandates new
  checks become 'their own thin action — not bolted into this one.'
  Frontmatter validation is also klappy.dev-specific: the enum values
  belong to one knowledge base, not to oddkit's generic codebase. Per
  Vodka Architecture, klappy.dev-specific rules belong here. The
  oddkit_audit job runs in parallel and is unaffected.

Tests:

  - scripts/tests/test_validator.py exercises all five rule classes
    against fixtures + the live writings/ directory (must remain clean
    for the test suite to pass).
  - Fixtures isolated under scripts/tests/fixtures/, outside the
    workflow's writings/ scan path.

Verification:

  $ python3 scripts/validate-frontmatter.py
  ✅ Frontmatter OK — 40 file(s) scanned, 0 findings.

  $ python3 scripts/tests/test_validator.py
  All validator smoke tests passed.

Canon:
  - klappy://canon/constraints/frontmatter-validation-before-merge
    (amended Automation section to describe the implementation)
  - klappy://canon/meta/frontmatter-schema (unchanged — source of truth)
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 10, 2026

Canon Quality — Frontmatter Schema ✅

All 40 file(s) in writings/ conform to klappy://canon/meta/frontmatter-schema.

Validator: scripts/validate-frontmatter.py · Canon: klappy://canon/constraints/frontmatter-validation-before-merge · Run: #123

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 10, 2026

Canon Quality — oddkit_audit

No dead klappy:// references or legacy link patterns found in writings/. 41 files scanned.

Spec: klappy://docs/oddkit/specs/oddkit-audit · Workflow: .github/workflows/canon-quality.yml · Run: #123

Comment thread scripts/validate-frontmatter.py Outdated
Comment thread scripts/validate-frontmatter.py Outdated
… contradiction

- Align ENUMS[audience] with canon/meta/frontmatter-schema.md
  (add operators, apocrypha; remove non-schema internal)
- Detect contradictory frontmatter even when public is the quoted
  string false, so both the type-mismatch and contradiction surface
  in a single CI pass
Comment thread scripts/validate-frontmatter.py Outdated
Comment thread scripts/validate-frontmatter.py
Python's bool is a subclass of int, so True == 1 and True in {1,2,3,4}
silently returns True. This let YAML 'tier: true' bypass both the enum
check and the integer type-mismatch check. Add an explicit bool-vs-int
guard in the enum validator so 'tier: true' is reported as an invalid
enum value.
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit f1c23b0. Configure here.

f"{field}: {v!r}",
f'Field "{field}" has value {v!r}, which is not in the '
f"allowed set: [{allowed_repr}]. Canon: {CANON_REF}",
))
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enum check emits misleading finding for quoted integers

Low Severity

When a user writes tier: "3", the enum check fires because the string "3" is not in {1, 2, 3, 4} (integers), emitting a frontmatter-invalid-enum finding saying the value is "not in the allowed set: [1, 2, 3, 4]." This is misleading — the value 3 IS valid, only the quoting is wrong. The frontmatter-type-mismatch check later correctly identifies the root cause. The author already added a bool_mismatch guard for True/False vs integer confusion in the same block, but the analogous string-digit case for INTEGER_FIELDS is unguarded, producing a confusing duplicate finding.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit f1c23b0. Configure here.

@klappy klappy merged commit 07721da into main May 11, 2026
3 checks passed
@klappy klappy deleted the feat/frontmatter-schema-validator branch May 11, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants