Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughA new PR review automation kit for Lamatic is added, including a Next.js frontend with a PR URL form, a server action that executes a Lamatic flow to analyze PR diffs fetched from GitHub, client components to render structured review JSON, and accompanying docs and config files. Changes
Sequence DiagramsequenceDiagram
actor User
participant Browser as Browser / Frontend
participant Server as Next.js Server
participant Lamatic as Lamatic Flow
participant GitHub as GitHub API
participant LLM as LLM
User->>Browser: Enter PR URL & submit
Browser->>Server: POST reviewPR(prUrl)
activate Server
Server->>Server: Validate PR URL
Server->>Lamatic: executeFlow(PR_REVIEW_FLOW_ID, { pr_url })
activate Lamatic
Lamatic->>Lamatic: Parse URL -> owner/repo/pr_number
Lamatic->>GitHub: GET /repos/{owner}/{repo}/pulls/{pr} (Accept: diff)
activate GitHub
GitHub-->>Lamatic: PR diff
deactivate GitHub
Lamatic->>LLM: Generate structured JSON (summary, issues, suggestions, verdict)
activate LLM
LLM-->>Lamatic: JSON response
deactivate LLM
Lamatic-->>Server: Return flow output
deactivate Lamatic
Server->>Server: Extract/parse review JSON
Server-->>Browser: PRReviewResult
deactivate Server
Browser->>Browser: Render verdict, summary, issues, suggestions, diffs
Browser-->>User: Display structured review
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Nitpick comments (4)
kits/automation/pr-review/.gitignore (1)
1-2: Consider expanding env/log patterns for safer defaults.You may want to also ignore additional local env variants and package-manager debug logs to prevent accidental commits across developer setups.
Optional hardening diff
.env .env.local +.env.*.local .next/ node_modules/ .DS_Store *.log +npm-debug.log* +yarn-debug.log* +pnpm-debug.log*Also applies to: 6-6
kits/automation/pr-review/.env.example (1)
1-4: Reduce dotenv-linter noise in the example env file.
If dotenv-linter is CI-gated, removing quotes and using a consistent key order avoids avoidable warnings.♻️ Suggested cleanup
-PR_REVIEW_FLOW_ID="YOUR_FLOW_ID" -LAMATIC_API_URL="YOUR_API_ENDPOINT" -LAMATIC_PROJECT_ID="YOUR_PROJECT_ID" -LAMATIC_API_KEY="YOUR_API_KEY" +LAMATIC_API_KEY=YOUR_API_KEY +LAMATIC_API_URL=YOUR_API_ENDPOINT +LAMATIC_PROJECT_ID=YOUR_PROJECT_ID +PR_REVIEW_FLOW_ID=YOUR_FLOW_IDkits/automation/pr-review/config.json (2)
9-15: Consider declaring all required runtime env vars in config metadata.
stepscurrently exposes onlyPR_REVIEW_FLOW_ID, but runtime also depends onLAMATIC_API_KEY,LAMATIC_PROJECT_ID, andLAMATIC_API_URL(seekits/automation/pr-review/lib/lamatic-client.ts, Lines 4-6). Surfacing them in kit config improves setup reliability.🧩 Suggested config extension
"steps": [ { "id": "pr-review-flow", "type": "mandatory", "envKey": "PR_REVIEW_FLOW_ID" + }, + { + "id": "lamatic-api-key", + "type": "mandatory", + "envKey": "LAMATIC_API_KEY" + }, + { + "id": "lamatic-project-id", + "type": "mandatory", + "envKey": "LAMATIC_PROJECT_ID" + }, + { + "id": "lamatic-api-url", + "type": "mandatory", + "envKey": "LAMATIC_API_URL" } ],
24-27: Fill public URLs (or use explicit TODO placeholders) before release.
LeavingdemoUrl,deployUrl, anddocumentationUrlblank makes the kit harder to validate and adopt.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: dc230a63-249e-4d9e-94f4-9e5f246fcfd1
⛔ Files ignored due to path filters (1)
kits/automation/pr-review/package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (13)
kits/automation/pr-review/.env.examplekits/automation/pr-review/.gitignorekits/automation/pr-review/README.mdkits/automation/pr-review/actions/orchestrate.tskits/automation/pr-review/app/layout.tsxkits/automation/pr-review/app/page.tsxkits/automation/pr-review/components/PRReviewForm.tsxkits/automation/pr-review/config.jsonkits/automation/pr-review/flows/pr-review-flow/README.mdkits/automation/pr-review/lib/lamatic-client.tskits/automation/pr-review/next-env.d.tskits/automation/pr-review/package.jsonkits/automation/pr-review/tsconfig.json
| if (!prUrl || !prUrl.includes("github.com")) { | ||
| throw new Error("Please provide a valid GitHub PR URL."); | ||
| } |
There was a problem hiding this comment.
Tighten PR URL validation.
Line 23 only checks includes("github.com"), so malformed/non-PR URLs can pass and fail later with less actionable errors.
🔍 Suggested validation fix
export async function reviewPR(prUrl: string): Promise<PRReviewResult> {
- if (!prUrl || !prUrl.includes("github.com")) {
- throw new Error("Please provide a valid GitHub PR URL.");
- }
+ if (!prUrl) throw new Error("Please provide a valid GitHub PR URL.");
+ let parsed: URL;
+ try {
+ parsed = new URL(prUrl);
+ } catch {
+ throw new Error("Please provide a valid GitHub PR URL.");
+ }
+ const isGitHubHost = parsed.hostname === "github.com" || parsed.hostname === "www.github.com";
+ const isPullRequestPath = /^\/[^/]+\/[^/]+\/pull\/\d+\/?$/.test(parsed.pathname);
+ if (!isGitHubHost || !isPullRequestPath) {
+ throw new Error("Please provide a valid GitHub PR URL.");
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (!prUrl || !prUrl.includes("github.com")) { | |
| throw new Error("Please provide a valid GitHub PR URL."); | |
| } | |
| if (!prUrl) throw new Error("Please provide a valid GitHub PR URL."); | |
| let parsed: URL; | |
| try { | |
| parsed = new URL(prUrl); | |
| } catch { | |
| throw new Error("Please provide a valid GitHub PR URL."); | |
| } | |
| const isGitHubHost = parsed.hostname === "github.com" || parsed.hostname === "www.github.com"; | |
| const isPullRequestPath = /^\/[^/]+\/[^/]+\/pull\/\d+\/?$/.test(parsed.pathname); | |
| if (!isGitHubHost || !isPullRequestPath) { | |
| throw new Error("Please provide a valid GitHub PR URL."); | |
| } |
| throw new Error("Could not parse review: " + review); | ||
| } |
There was a problem hiding this comment.
Avoid throwing raw flow payloads in error messages.
Lines 43 and 51 append raw model/flow output to thrown errors, which can leak large or sensitive response content and bloat error transport.
🔒 Suggested safer errors
- throw new Error("Could not parse review: " + review);
+ throw new Error("Could not parse review payload from flow response.");
...
- throw new Error("Could not find review in response: " + JSON.stringify(response));
+ throw new Error("Flow response did not include a valid review object.");Also applies to: 51-51
| if (review?.summary) { | ||
| return review as PRReviewResult; | ||
| } |
There was a problem hiding this comment.
Validate full review shape before returning to UI.
Line 47 only checks review.summary; if issues/suggestions are missing or non-arrays, the client will crash when reading .filter() / .length.
✅ Suggested shape guard
- if (review?.summary) {
+ if (
+ review &&
+ typeof review.summary === "string" &&
+ Array.isArray(review.issues) &&
+ Array.isArray(review.suggestions) &&
+ (review.verdict === "approve" || review.verdict === "needs_changes" || review.verdict === "discuss")
+ ) {
return review as PRReviewResult;
}| <CodePane label="BEFORE" symbol="-" text={code} tone="#ff000015" onCopy={copyFix} /> | ||
| <CodePane label="AFTER" symbol="+" text={fix} tone="#00ff0015" borderTop onCopy={copyFix} /> |
There was a problem hiding this comment.
Fix copy behavior: BEFORE pane currently copies the fix.
Both panes call copyFix, so clicking Copy on the “BEFORE” block copies the wrong content.
📋 Suggested fix
function CodeDiff({ code, fix }: { code: string; fix: string }) {
+ async function copyCode() {
+ try {
+ await navigator.clipboard.writeText(code);
+ } catch {
+ // no-op
+ }
+ }
+
async function copyFix() {
try {
await navigator.clipboard.writeText(fix);
@@
- <CodePane label="BEFORE" symbol="-" text={code} tone="#ff000015" onCopy={copyFix} />
+ <CodePane label="BEFORE" symbol="-" text={code} tone="#ff000015" onCopy={copyCode} />
<CodePane label="AFTER" symbol="+" text={fix} tone="#00ff0015" borderTop onCopy={copyFix} />📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| <CodePane label="BEFORE" symbol="-" text={code} tone="#ff000015" onCopy={copyFix} /> | |
| <CodePane label="AFTER" symbol="+" text={fix} tone="#00ff0015" borderTop onCopy={copyFix} /> | |
| function CodeDiff({ code, fix }: { code: string; fix: string }) { | |
| async function copyCode() { | |
| try { | |
| await navigator.clipboard.writeText(code); | |
| } catch { | |
| // no-op | |
| } | |
| } | |
| async function copyFix() { | |
| try { | |
| await navigator.clipboard.writeText(fix); | |
| } catch { | |
| // no-op | |
| } | |
| } | |
| return ( | |
| <> | |
| <CodePane label="BEFORE" symbol="-" text={code} tone="#ff000015" onCopy={copyCode} /> | |
| <CodePane label="AFTER" symbol="+" text={fix} tone="#00ff0015" borderTop onCopy={copyFix} /> | |
| </> | |
| ); | |
| } |
| ) : !result || !verdict ? ( | ||
| <EmptyLeftPanel /> | ||
| ) : ( |
There was a problem hiding this comment.
Show errors when review generation fails.
When reviewPR throws, result stays null, and Line 565 routes to EmptyLeftPanel, so the error set at Line 627 is never rendered.
🛠️ Suggested rendering fix
- ) : !result || !verdict ? (
+ ) : error ? (
+ <div style={{ fontSize: 13, color: "#fca5a5" }}>{error}</div>
+ ) : !result || !verdict ? (
<EmptyLeftPanel />
) : (Also applies to: 627-627
| This flow has 4 nodes: **API Request → Code → API → Generate JSON → API Response** | ||
|
|
There was a problem hiding this comment.
Fix node count mismatch in flow description.
Line 17 says “4 nodes,” but the pipeline shown includes 5 nodes (API Request → Code → API → Generate JSON → API Response).
| ``` | ||
| You are a senior software engineer doing a thorough code review. You will receive a unified diff from a GitHub pull request. | ||
|
|
||
| Return ONLY a valid JSON object with this exact shape: | ||
| { | ||
| "summary": "3-4 sentences describing what this PR does, why it exists, and what files/systems it touches", | ||
| "issues": [ | ||
| { | ||
| "severity": "CRITICAL", | ||
| "file": "src/auth.ts", | ||
| "line": 42, | ||
| "description": "specific description of the problem and WHY it is dangerous", | ||
| "code": "exact snippet from the diff that is problematic (max 3 lines)", | ||
| "fix": "exact replacement code, ready to copy-paste (max 5 lines)", | ||
| "fix_explanation": "one sentence explaining what the fix does and why it solves the problem" | ||
| } | ||
| ], | ||
| "suggestions": [ | ||
| { | ||
| "type": "PERF", | ||
| "file": "src/utils.ts", | ||
| "line": 18, | ||
| "description": "specific improvement and its impact", | ||
| "code": "current code snippet (max 3 lines)", | ||
| "fix": "improved version, ready to copy-paste (max 5 lines)", | ||
| "fix_explanation": "one sentence explaining the improvement" | ||
| } | ||
| ], | ||
| "verdict": "approve" | ||
| } | ||
|
|
||
| Rules: | ||
| - verdict must be one of: "approve", "needs_changes", "discuss" | ||
| - severity must be one of: "CRITICAL", "WARNING", "INFO" | ||
| - type must be one of: "PERF", "STYLE", "TEST", "DOCS" | ||
| - fix field MUST be valid, working, copy-pasteable code — not a description of what to do | ||
| - fix_explanation must be plain English, not code | ||
| - file and line must reference actual files and line numbers from the diff | ||
| - If no issues or suggestions exist, use empty arrays | ||
| - No markdown, no code fences, no explanation outside the JSON. Just the raw JSON object. | ||
| ``` |
There was a problem hiding this comment.
Add a language identifier to the fenced code block.
This fence is missing a language tag, which triggers markdownlint MD040.
🧹 Suggested markdown fix
-```
+```text
You are a senior software engineer doing a thorough code review. You will receive a unified diff from a GitHub pull request.
...
-```
+```📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ``` | |
| You are a senior software engineer doing a thorough code review. You will receive a unified diff from a GitHub pull request. | |
| Return ONLY a valid JSON object with this exact shape: | |
| { | |
| "summary": "3-4 sentences describing what this PR does, why it exists, and what files/systems it touches", | |
| "issues": [ | |
| { | |
| "severity": "CRITICAL", | |
| "file": "src/auth.ts", | |
| "line": 42, | |
| "description": "specific description of the problem and WHY it is dangerous", | |
| "code": "exact snippet from the diff that is problematic (max 3 lines)", | |
| "fix": "exact replacement code, ready to copy-paste (max 5 lines)", | |
| "fix_explanation": "one sentence explaining what the fix does and why it solves the problem" | |
| } | |
| ], | |
| "suggestions": [ | |
| { | |
| "type": "PERF", | |
| "file": "src/utils.ts", | |
| "line": 18, | |
| "description": "specific improvement and its impact", | |
| "code": "current code snippet (max 3 lines)", | |
| "fix": "improved version, ready to copy-paste (max 5 lines)", | |
| "fix_explanation": "one sentence explaining the improvement" | |
| } | |
| ], | |
| "verdict": "approve" | |
| } | |
| Rules: | |
| - verdict must be one of: "approve", "needs_changes", "discuss" | |
| - severity must be one of: "CRITICAL", "WARNING", "INFO" | |
| - type must be one of: "PERF", "STYLE", "TEST", "DOCS" | |
| - fix field MUST be valid, working, copy-pasteable code — not a description of what to do | |
| - fix_explanation must be plain English, not code | |
| - file and line must reference actual files and line numbers from the diff | |
| - If no issues or suggestions exist, use empty arrays | |
| - No markdown, no code fences, no explanation outside the JSON. Just the raw JSON object. | |
| ``` |
🧰 Tools
🪛 markdownlint-cli2 (0.21.0)
[warning] 82-82: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
| export const lamatic = new Lamatic({ | ||
| apiKey: process.env.LAMATIC_API_KEY!, | ||
| projectId: process.env.LAMATIC_PROJECT_ID!, | ||
| endpoint: process.env.LAMATIC_API_URL!, | ||
| }); |
There was a problem hiding this comment.
Replace non-null assertions with explicit env validation.
Current initialization can crash at import time with poor diagnostics when any required variable is missing.
🛠️ Suggested fix
import { Lamatic } from "lamatic";
+function requiredEnv(name: "LAMATIC_API_KEY" | "LAMATIC_PROJECT_ID" | "LAMATIC_API_URL"): string {
+ const value = process.env[name];
+ if (!value) {
+ throw new Error(`Missing required environment variable: ${name}`);
+ }
+ return value;
+}
+
export const lamatic = new Lamatic({
- apiKey: process.env.LAMATIC_API_KEY!,
- projectId: process.env.LAMATIC_PROJECT_ID!,
- endpoint: process.env.LAMATIC_API_URL!,
+ apiKey: requiredEnv("LAMATIC_API_KEY"),
+ projectId: requiredEnv("LAMATIC_PROJECT_ID"),
+ endpoint: requiredEnv("LAMATIC_API_URL"),
});📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export const lamatic = new Lamatic({ | |
| apiKey: process.env.LAMATIC_API_KEY!, | |
| projectId: process.env.LAMATIC_PROJECT_ID!, | |
| endpoint: process.env.LAMATIC_API_URL!, | |
| }); | |
| import { Lamatic } from "lamatic"; | |
| function requiredEnv(name: "LAMATIC_API_KEY" | "LAMATIC_PROJECT_ID" | "LAMATIC_API_URL"): string { | |
| const value = process.env[name]; | |
| if (!value) { | |
| throw new Error(`Missing required environment variable: ${name}`); | |
| } | |
| return value; | |
| } | |
| export const lamatic = new Lamatic({ | |
| apiKey: requiredEnv("LAMATIC_API_KEY"), | |
| projectId: requiredEnv("LAMATIC_PROJECT_ID"), | |
| endpoint: requiredEnv("LAMATIC_API_URL"), | |
| }); |
| ## Demo | ||
|
|
||
| > | ||
|
|
There was a problem hiding this comment.
Populate or remove the empty Demo section.
Line 20 contains only an empty quote marker, so the “Demo” section renders blank and looks unfinished.
📝 Suggested doc fix
## Demo
->
+Add a screenshot/GIF or live demo URL here.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ## Demo | |
| > | |
| ## Demo | |
| Add a screenshot/GIF or live demo URL here. |
|
Missing Lamatic Flow Configs |
3 participants
What This Kit Does
AI-powered GitHub pull request reviewer. Paste any public PR URL and get an instant structured code review, with file-level citations, before/after code diffs, and actionable fixes ready to copy-paste.
What it returns:
approve,needs_changes, ordiscussWhy this is useful: Teams using Lamatic can drop this into their dev workflow to get consistent, structured reviews on every PR, without waiting for a senior engineer to be available.
Providers & Prerequisites
GITHUB_TOKENto your env and pass it asAuthorization: Bearerin the API node header.How to Run Locally
cd kits/automation/pr-reviewnpm installcp .env.example .envand fill in valuesnpm run devOpen http://localhost:3000, paste a GitHub PR URL, hit Review PR.
Full flow setup instructions:
flows/pr-review-flow/README.mdLive Preview
https://judgethepr.vercel.app/
Lamatic Flow
Flow ID:
a70ea776-386d-4474-953c-a7fe7bf54d41Built with: API Request → Code (URL parser) → API (GitHub diff fetch) → Generate JSON (LLM review) → API Response
Checklist
npm run dev.env.examplehas no secrets, only placeholdersREADME.mddocuments setup and usagekits/automation/pr-review/config.jsonis present and validflows/folder