feat(hook): add fact-checking PreToolUse hook for Write/Edit tools#83
feat(hook): add fact-checking PreToolUse hook for Write/Edit tools#83
Conversation
Advisory hook that warns when Write/Edit tools are invoked without evidence of prior file reading. The hook checks tool input for research indicators (grep results, line references, etc.) and emits a stderr warning that gets injected into LLM context. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
|
The following comment was made by an LLM, it may be inaccurate: |
There was a problem hiding this comment.
Pull request overview
Adds an advisory PreToolUse hook (Claude Code parity) to warn when write/edit tools are invoked without textual “evidence” of prior file reading, and introduces tests to validate the hook behavior.
Changes:
- Add
.opencode/hooks/enforce-factcheck-before-edit.shfact-checking hook (advisory-only, stderr warning). - Register the hook in
.opencode/opencode.jsoncfor PreToolUse onwriteandedit. - Add Bun tests covering warning vs. evidence-detected cases.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
packages/opencode/test/hook/factcheck.test.ts |
Adds unit tests for the new fact-checking hook script behavior. |
.opencode/opencode.jsonc |
Registers the new PreToolUse hook for write/edit. |
.opencode/hooks/enforce-factcheck-before-edit.sh |
Implements advisory heuristic detection + warning message for edits without “evidence”. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "command": ".opencode/hooks/enforce-factcheck-before-edit.sh", | ||
| "matcher": "write", | ||
| }, | ||
| { | ||
| "command": ".opencode/hooks/enforce-factcheck-before-edit.sh", |
There was a problem hiding this comment.
The hook command is a relative path and relies on the script being executable and the process CWD being the repo root. Since hooks are executed via sh -c, it’s more robust to invoke it explicitly (e.g., sh "$OPENCODE_PROJECT_DIR/.opencode/hooks/enforce-factcheck-before-edit.sh") so it works regardless of CWD/exec-bit preservation.
| "command": ".opencode/hooks/enforce-factcheck-before-edit.sh", | |
| "matcher": "write", | |
| }, | |
| { | |
| "command": ".opencode/hooks/enforce-factcheck-before-edit.sh", | |
| "command": "sh \"$OPENCODE_PROJECT_DIR/.opencode/hooks/enforce-factcheck-before-edit.sh\"", | |
| "matcher": "write", | |
| }, | |
| { | |
| "command": "sh \"$OPENCODE_PROJECT_DIR/.opencode/hooks/enforce-factcheck-before-edit.sh\"", |
| { | ||
| "command": ".opencode/hooks/enforce-factcheck-before-edit.sh", | ||
| "matcher": "edit", | ||
| }, |
There was a problem hiding this comment.
This PreToolUse hook only matches write/edit, but OpenCode can disable those tools in favor of apply_patch (and edits can also happen via multiedit). In those cases the fact-check hook won’t run at all, which undermines the intended coverage; consider adding matchers (and corresponding script support) for the other edit-capable tools.
| }, | |
| }, | |
| { | |
| "command": ".opencode/hooks/enforce-factcheck-before-edit.sh", | |
| "matcher": "apply_patch", | |
| }, | |
| { | |
| "command": ".opencode/hooks/enforce-factcheck-before-edit.sh", | |
| "matcher": "multiedit", | |
| }, |
| # Check if tool input contains evidence of prior file reading | ||
| if echo "$INPUT" | grep -qiE '(based on reading|as shown in|from the file|grep result|read tool output|line [0-9]+)'; then | ||
| echo "FACT-CHECK: Evidence of prior research detected" >&2 |
There was a problem hiding this comment.
Consider using printf '%s' "$INPUT" instead of echo "$INPUT" for POSIX portability and to avoid any shell-specific echo edge cases (even though JSON usually starts with {).
|
|
||
| # Check if tool input contains evidence of prior file reading | ||
| if echo "$INPUT" | grep -qiE '(based on reading|as shown in|from the file|grep result|read tool output|line [0-9]+)'; then | ||
| echo "FACT-CHECK: Evidence of prior research detected" >&2 |
There was a problem hiding this comment.
On evidence detected, the script emits a success message to stderr. Hook stderr is surfaced as HookResult.message even on pass, so this will add extra context/noise (and token usage) to every Write/Edit that contains any evidence phrase. Consider staying silent on success and only emitting stderr for warnings (or guard the success message behind a debug flag).
| echo "FACT-CHECK: Evidence of prior research detected" >&2 |
| test("warns on Write without evidence", async () => { | ||
| const result = await runHook( | ||
| { command: SCRIPT_PATH }, | ||
| makeEnv("Write", '{"file_path":"/tmp/test.ts","content":"hello"}'), | ||
| ) | ||
| expect(result.action).toBe("pass") | ||
| expect(result.message).toContain("FACT-CHECK WARNING") | ||
| }) | ||
|
|
||
| test("warns on edit without evidence", async () => { | ||
| const result = await runHook( | ||
| { command: SCRIPT_PATH }, | ||
| makeEnv("edit", '{"old_string":"foo","new_string":"bar"}'), | ||
| ) |
There was a problem hiding this comment.
These tests use snake_case tool input keys (e.g., file_path, old_string, new_string), but the actual tool schemas use camelCase (filePath, oldString, newString). Using the real shapes would make the tests less misleading and more resilient if the hook later inspects specific fields (like filePath).
| # Only apply to write/edit tools | ||
| case "$TOOL" in | ||
| write|Write|edit|Edit) ;; |
There was a problem hiding this comment.
The script itself only applies to write|Write|edit|Edit. If you expand config matchers to include other edit-capable tools (e.g., apply_patch, multiedit), this case statement will also need to include them to ensure the hook actually runs for those tool invocations.
| # Only apply to write/edit tools | |
| case "$TOOL" in | |
| write|Write|edit|Edit) ;; | |
| # Only apply to edit-capable tools | |
| case "$TOOL" in | |
| write|Write|edit|Edit|apply_patch|ApplyPatch|applypatch|Applypatch|multiedit|multiEdit) ;; |
…ches (#83) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
.opencode/hooks/enforce-factcheck-before-edit.shを新規作成（POSIX shell、advisory only）.opencode/opencode.jsoncに hooks.PreToolUse matcher 登録What
Claude Code parity:
enforce-factcheck-before-edit.sh相当のfact-checking hookをOpenCodeに追加。Type
Verify
bun test test/hook/— 44テスト全パスbun typecheck— 13パッケージ全パスOPENCODE_TOOL_NAME=Write OPENCODE_TOOL_INPUT='{}' sh .opencode/hooks/enforce-factcheck-before-edit.sh→ WARNING出力確認Checklist
Issue
Closes #53
🤖 Generated with Claude Code