ci: add test coverage ratchet to prevent regression

[cv]

Summary

Add a CI mechanism that tracks test coverage and prevents it from decreasing, while automatically raising the bar when coverage improves.

Motivation

NemoClaw has two test suites (root-level JS tests and TypeScript Vitest tests in nemoclaw/), but no coverage gate in CI. Without one, PRs can merge code with zero tests and silently erode overall coverage. A coverage ratchet enforces a floor that only goes up — contributors are not asked to hit an arbitrary high bar, but they cannot make things worse.

Specification

How the ratchet works

1. A threshold file is checked into the repo (e.g., ci/coverage-threshold.txt) containing a single number — the current minimum acceptable coverage percentage. For example: 65.0.

2. During CI, after running tests with coverage enabled, a script:

   • Reads the current coverage percentage from the test output.
   • Reads the threshold from the threshold file.
   • Fails the build if coverage is below (threshold - tolerance). A tolerance of 1% accounts for platform differences and floating-point variation.
   • If coverage exceeds the threshold, the script prints a message noting the improvement. Optionally, it can update the threshold file and commit it back (or leave that to the contributor to do manually).

3. The threshold file is the single source of truth. Maintainers can manually lower it in exceptional cases (with a commit message explaining why), but normal workflow only ever raises it.

Implementation for both test suites

Root-level tests (node --test test/*.test.js)

• Node.js 22+ has built-in coverage via --experimental-test-coverage flag.
• Run: node --test --experimental-test-coverage test/*.test.js
• Parse the summary line for the total coverage percentage.

TypeScript tests (nemoclaw/ — Vitest)

• Vitest supports coverage via --coverage flag with @vitest/coverage-v8 or @vitest/coverage-istanbul.
• Run: npx vitest run --coverage
• Parse the coverage summary from stdout or the JSON report file.

CI integration

Add steps to the existing pr.yaml workflow:

- name: Run root unit tests with coverage
  run: node --test --experimental-test-coverage test/*.test.js 2>&1 | tee coverage-root.txt

- name: Run TypeScript unit tests with coverage
  working-directory: nemoclaw
  run: npx vitest run --coverage 2>&1 | tee coverage-ts.txt

- name: Check coverage ratchet
  run: |
    # Script that:
    # 1. Extracts coverage % from both reports
    # 2. Compares against ci/coverage-threshold.txt
    # 3. Fails if below threshold - tolerance
    # 4. Prints congratulations if above threshold
    scripts/check-coverage-ratchet.sh

Bootstrapping

To set the initial threshold:

1. Run both test suites with coverage locally.
2. Round down the current coverage to the nearest integer.
3. Write that number to ci/coverage-threshold.txt.
4. Commit it. From that point on, coverage can only go up.

Acceptance criteria

• A threshold file exists in the repo with the current coverage floor.
• CI runs tests with coverage enabled for both test suites.
• CI fails if coverage drops below the threshold (with tolerance).
• When coverage improves, the output clearly notes the new high-water mark.
• The threshold file is the only thing that needs updating — no workflow edits required to raise the bar.

[prekshivyas]

Fixed in PR #522 (commit 61674ae), which added test coverage reporting and thresholds on the same day this issue was filed. Subsequently improved in PR #913 (bash→TypeScript rewrite) and PR #1131 (CI workflow consolidation).

Coverage ratchet is fully operational:

• Threshold files: ci/coverage-threshold-cli.json and ci/coverage-threshold-plugin.json
• Check script: scripts/check-coverage-ratchet.ts
• Enforcement: pre-commit hooks + CI workflow