feat(plugins): add edit-verifier plugin for post-edit verification

[mvanhorn]

Summary

Addresses #32658

When Claude uses the Edit tool, it assumes the edit succeeded without verifying. If the target text wasn't found (whitespace mismatch, code already modified), the edit silently fails and Claude proceeds with incorrect assumptions.

This plugin adds a PostToolUse hook that reads the file back after each Edit operation and checks that the new_string is present. If not found, it warns Claude via systemMessage to re-read the file.

Design

• Non-blocking: Warns only, never denies operations
• Lightweight: Reads one file per edit (the file that was just edited)
• Smart skipping: Ignores very short replacements (< 5 chars) to avoid false positives
• Graceful errors: If the file can't be read, warns instead of crashing

Plugin structure

plugins/edit-verifier/
  .claude-plugin/plugin.json
  hooks/hooks.json          # PostToolUse hook for Edit tool
  hooks/post_edit_verify.py # Verification logic
  README.md

Test plan

• Edit a file successfully - no warning should appear
• Edit a file with a new_string that doesn't match (e.g., wrong whitespace) - warning should appear
• Edit a nonexistent file - graceful warning about unable to read
• Very short edits (< 5 chars) - skipped, no verification

This contribution was developed with AI assistance (Claude Code).

[VoxCore84]

Hey @mvanhorn -- great plugin, we've been running a similar hook in our project. Here's a merged version of post_edit_verify.py that layers three enhancements on top of your implementation. I kept your plugin structure, function flow, and default behavior (warn-only, min 5 chars) so nothing is a breaking change.

What's added

1. Configurable minimum threshold (EDIT_VERIFY_MIN_CHARS env var)

Your version hardcodes < 5. This makes it an env var (still defaults to 5) so users can tune it without forking.

2. Old-string-gone check (catches wrong-occurrence replacements)

Your hook checks that new_string is present -- but doesn't verify that old_string is gone. If a file has two identical blocks and Claude edits the wrong one, new_string lands in the wrong location and the original target is untouched. This adds an old_string in content check.

3. False-alarm reduction

The old-string check only fires when new_string is also missing. If old_string legitimately appears in multiple places (imports, repeated patterns), a successful edit will have new_string present and we skip the alarm. This prevents noise from common code patterns.

4. Configurable mode (EDIT_VERIFY_MODE env var)

warn (default) uses systemMessage matching your original behavior. block uses decision: block which pauses Claude for review -- useful for CI/automated pipelines where you want hard stops on verification failures.

5. Encoding fallback chain

Tries utf-8 -> system default -> latin1 so the hook doesn't crash on files with non-UTF-8 encoding (common in legacy codebases).

Other cleanup

• Removed replace_all code path -- this parameter doesn't exist in Claude Code's Edit tool schema, so it was dead code that could draw review friction.
• Already using python3 in hooks.json (your version does too -- no change needed there).
• Added a tool_response.success pre-check: if the Edit tool itself already reported failure, we skip verification since there's nothing to verify.

The merged file

Full hooks/post_edit_verify.py (click to expand)

#!/usr/bin/env python3
"""Post-edit verification hook for Claude Code.

After an Edit tool operation, reads the file back and checks that the
new_string is present AND that old_string is gone. If verification fails,
either warns (systemMessage) or blocks (decision: block) based on config.

Original implementation by mvanhorn (PR #32755).
Enhancements by VoxCore84:
  1. Configurable minimum threshold via EDIT_VERIFY_MIN_CHARS env var
  2. Checks that old_string is gone after edit (catches wrong-occurrence edits)
  3. False-alarm reduction: only flags old_string presence when new_string
     is also missing (prevents false positives from legitimate duplicates)
  4. Configurable mode via EDIT_VERIFY_MODE env var (warn|block)
  5. Encoding fallback chain for cross-platform file reads

Environment variables:
  EDIT_VERIFY_MIN_CHARS  Minimum stripped length of new_string to verify
                         (default: 5, matching original behavior)
  EDIT_VERIFY_MODE       "warn" uses systemMessage (default, non-blocking)
                         "block" uses decision:block (pauses Claude for review)
"""

import json
import os
import sys


def main():
    try:
        input_data = json.load(sys.stdin)
    except (json.JSONDecodeError, EOFError):
        sys.exit(0)

    tool_name = input_data.get("tool_name", "")
    if tool_name != "Edit":
        json.dump({}, sys.stdout)
        sys.exit(0)

    tool_input = input_data.get("tool_input", {})
    file_path = tool_input.get("file_path", "")
    old_string = tool_input.get("old_string", "")
    new_string = tool_input.get("new_string", "")

    # --- Configuration from environment ---
    min_chars = int(os.environ.get("EDIT_VERIFY_MIN_CHARS", "5"))
    mode = os.environ.get("EDIT_VERIFY_MODE", "warn").lower()

    # Skip verification for empty edits or very short replacements
    if not file_path or not new_string or len(new_string.strip()) < min_chars:
        json.dump({}, sys.stdout)
        sys.exit(0)

    # Skip if the Edit tool already reported failure
    tool_response = input_data.get("tool_response", {})
    if isinstance(tool_response, dict) and not tool_response.get("success", True):
        json.dump({}, sys.stdout)
        sys.exit(0)

    # --- Read the file back with encoding fallback chain ---
    content = None
    for encoding in ("utf-8", None, "latin1"):
        try:
            with open(file_path, "r", encoding=encoding) as f:
                content = f.read()
            break
        except (UnicodeDecodeError, UnicodeError):
            continue
        except (FileNotFoundError, PermissionError, OSError):
            msg = (
                f"Edit verification: Could not read {file_path} to verify "
                f"the edit applied correctly. Please verify manually."
            )
            json.dump(_make_result(msg, mode), sys.stdout)
            sys.exit(0)

    if content is None:
        msg = (
            f"Edit verification: Could not decode {file_path} with any "
            f"supported encoding (utf-8, system default, latin1). "
            f"Please verify the edit manually."
        )
        json.dump(_make_result(msg, mode), sys.stdout)
        sys.exit(0)

    # --- Verify the edit applied correctly ---
    problems = []

    # Check 1: new_string should be present in the file
    new_string_found = new_string in content
    if not new_string_found:
        problems.append(
            f"Expected new content was not found in {file_path} after the "
            f"Edit operation."
        )

    # Check 2: old_string should be gone (with false-alarm reduction)
    #
    # Only flag old_string still present when new_string is ALSO missing.
    # This handles the common case where old_string legitimately appears in
    # multiple locations -- if the edit succeeded on the target occurrence,
    # new_string will be present and we skip the alarm.
    if old_string and old_string != new_string and old_string in content:
        if not new_string_found:
            occurrences = content.count(old_string)
            problems.append(
                f"Original content still appears {occurrences} time(s) in "
                f"{file_path} and the replacement text is missing. The edit "
                f"may have targeted the wrong occurrence or failed to apply."
            )

    # --- Report results ---
    if problems:
        detail = " ".join(problems)
        msg = (
            f"Edit verification warning: {detail} "
            f"Please read the file to verify."
        )
        json.dump(_make_result(msg, mode), sys.stdout)
    else:
        # Edit verified successfully - silent
        json.dump({}, sys.stdout)


def _make_result(message, mode):
    """Build the hook result dict based on configured mode.

    - "warn" (default): non-blocking systemMessage
    - "block": pauses Claude with a decision:block prompt
    """
    if mode == "block":
        return {"decision": "block", "reason": message}
    else:
        return {"systemMessage": message}


if __name__ == "__main__":
    main()

How to apply

You can either:

• Replace your hooks/post_edit_verify.py with the version above (drop-in, no changes to plugin.json or hooks.json needed)
• Or I can open a PR against your mvanhorn:edit-verifier branch -- let me know

No changes needed to hooks.json or plugin.json. The README would benefit from a "Configuration" section documenting the two env vars, happy to draft that too if you want.

[mvanhorn]

@VoxCore84 These are solid improvements. The old-string-gone check with false-alarm reduction is clever - catching wrong-occurrence edits without noisy false positives from legitimate duplicates. The encoding fallback and configurable mode are practical additions too.

Happy to incorporate these. I'll update the PR with your merged version and add the Configuration section to the README. Will credit you in the commit.

[VoxCore84]

Any update on this? @mvanhorn wrote a working edit-verifier plugin back on March 10 in response to #32658 — it's been 17 days with no review.

This is exactly the kind of community contribution that should get fast feedback. The core issue (Edit tool doesn't verify its own output) is still unfixed and still reproducing daily.

[mvanhorn]

Yeah, waiting on maintainer review here too. Nothing actionable on our end.