feat: wire rivet mechanical oracle into AI review#28
Merged
Conversation
## Why PR #26 shipped the rivet-oracle module as a building block. This PR makes it actually run during PR review for repos that ship rivet.yaml. The result: name-anchored, mechanically-validated findings produced *without* the model's involvement, prepended to whatever the model produces. This realises the "model proposes, oracle decides" pattern. Oracle findings: - bypass the slop filter (mechanically validated by construction) - bypass quote-or-die (anchored to artifact_id, not file:line) - promote the verdict to `request_changes` when severity = error Result on rivet PRs: a review that has actual evidence-grade findings even if Ollama times out, hallucinates, or returns empty findings. ## What ### `src/rivet-fetch.js` (new) - `hasRivetYaml(octokit, owner, repo, ref)` — cheap pre-check via Contents API. Returns false on 404 / auth errors. Skipping the tarball when this returns false saves ~50 MB per PR for non-instrumented repos. - `fetchAndExtractTarball(...)` — pulls `GET /tarball/{ref}`, pipes through `tar -xz --strip-components=1` into a tempdir. - `withTempRepoCheckout(...)` — orchestrator that creates the tempdir, fetches+extracts, runs the caller's callback, and cleans up (even on error). ### `src/ai-review.js` Before the model call, run the oracle if (a) `rivet_oracle.enabled`, (b) `binary_path` configured, (c) `rivet.yaml` exists at PR head. Failures are non-fatal and logged at warn — the model path still runs. Oracle findings are merged into the rendered comment. ### `src/ai-review-prompt.js` - `computeVerdict()` now promotes to `request_changes` when any oracle finding has severity `error` (broken cross-refs, missing required fields). - `renderReviewMarkdown()` formats oracle findings differently from model findings: severity-emoji + `artifact_id` + oracle name, vs file:line + quoted_line for model findings. ### `config.yml` New `rivet_oracle:` section: ```yaml rivet_oracle: enabled: true binary_path: "data/rivet/rivet" timeout_ms: 60000 ``` Schema validated. ## What's NOT here - The rivet binary on netcup. Manual install (one shot) until PR-D ships the auto-installer. - Cache for tarball / oracle results across reviews on the same SHA. - Integration test that spawns rivet for real (would need the binary in CI; today's tests stub the runner). ## Test plan - [x] 766 tests pass (was 753 — added 13 covering tarball fetch, has-yaml pre-check, severity-error verdict promotion, oracle finding render format, mixed oracle+model output) - [x] eslint clean - [ ] **After deploy + manual rivet install on netcup**: open a small PR against `pulseengine/rivet` and verify the bot's review contains an oracle-validated finding (e.g. `🔴 SPAR-REQ-001 (rivet-validate)`). - [ ] Verify temper PRs (no rivet.yaml) still get reviews with no oracle section — should be unchanged from #24's behaviour. ## Risk & rollout - Risk: medium. New code paths run only when binary exists at the configured path; without it, the oracle silently skips. So shipping this before installing the binary is safe — the behaviour reverts to PR #24's model-only review. - Rollout: self-update on merge. Then SCP the rivet-v0.4.3 Linux binary to `/opt/temper/data/rivet/rivet` and `chmod +x`. After that, the next rivet PR opened gets oracle-validated findings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Author
|
/review-pr |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
PR #26 shipped the rivet-oracle module as a building block. This PR makes it actually run during PR review for repos that ship `rivet.yaml`. The result: name-anchored, mechanically-validated findings produced without the model's involvement, prepended to whatever the model produces.
This realises the "model proposes, oracle decides" pattern. Oracle findings:
Result on rivet PRs: a review with actual evidence-grade findings even if Ollama times out, hallucinates, or returns empty findings.
What
`src/rivet-fetch.js` (new)
`src/ai-review.js`
Before the model call, run the oracle if (a) `rivet_oracle.enabled`, (b) `binary_path` configured, (c) `rivet.yaml` at PR head. Failures non-fatal — model path still runs. Oracle findings merged into rendered comment.
`src/ai-review-prompt.js`
Config
New `rivet_oracle:` section in `config.yml` with schema validation.
What's NOT here
Test plan
Risk & rollout
🤖 Generated with Claude Code