Skip to content

test(gadgets): add unit tests for FileSearchAndReplace#643

Merged
zbigniewsobiecki merged 2 commits intodevfrom
feature/file-search-and-replace-tests
Mar 7, 2026
Merged

test(gadgets): add unit tests for FileSearchAndReplace#643
zbigniewsobiecki merged 2 commits intodevfrom
feature/file-search-and-replace-tests

Conversation

@aaight
Copy link
Copy Markdown
Collaborator

@aaight aaight commented Mar 7, 2026

Summary

  • Adds tests/unit/gadgets/fileSearchAndReplace.test.ts with 18 test cases covering all acceptance criteria
  • Follows the same mock pattern as fileInsertContent.test.ts (mocking pathValidation, readTracking, postEditChecks, and diagnosticState)
  • Uses temp directories for real filesystem operations so replacement behavior is verified end-to-end

Test coverage added

  • Exact match — single replacement, file path in output, before/after context display
  • Whitespace-tolerant match — tabs in file vs spaces in search pattern
  • replaceAll=true — replaces all occurrences, output includes match count and line ranges
  • expectedCount mismatch — aborts with clear error listing found locations; passes when count matches
  • No match found — throws with NOT FOUND error including the search content
  • Multiple matches without replaceAll — throws Ambiguous error mentioning replaceAll option and match count
  • File not found — throws File not found error
  • Empty replace (deletion) — single deletion and replaceAll deletion both verified; replaceAll output includes "All matches deleted."

Test plan

  • npx vitest run tests/unit/gadgets/fileSearchAndReplace.test.ts — 18/18 pass
  • npx vitest run --project unit — 240/240 pass (no regressions)

Card: https://trello.com/c/TwTKy2Tj/179-as-a-developer-i-want-unit-tests-for-the-filesearchandreplace-gadget-so-that-the-most-used-edit-tool-has-regression-coverage

🤖 Generated with Claude Code

@nhopeatall
Copy link
Copy Markdown
Collaborator

nhopeatall commented Mar 7, 2026

✅ Review complete for PR #643. Solid test coverage — 18 tests covering all key behaviors of FileSearchAndReplace. One inline comment about a tautological assertion in the 'provides suggestions' test that should be tightened to actually verify the behavior it claims to test.

Copy link
Copy Markdown
Collaborator

@nhopeatall nhopeatall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Solid test coverage for the FileSearchAndReplace gadget. One test has a weak assertion worth tightening; otherwise LGTM.

Code Issues

Should Fix

  • tests/unit/gadgets/fileSearchAndReplace.test.ts:260 — The 'provides suggestions when similar content exists' test has a tautological assertion: expect(errorMessage.length > 0 || readFileSync(...).includes('5000')).toBe(true). This passes whether the operation throws OR succeeds via fuzzy matching — it's asserting "something happened" rather than verifying the suggestions behavior the test name promises. If the intent is to verify suggestions appear in the error, assert on the error message content (e.g., expect(errorMessage).toMatch(/similar|did you mean/i)). If the intent is to verify fuzzy matching succeeds, assert on the file content directly. The current form won't catch regressions in either path.

// The important thing is it doesn't silently do nothing
expect(errorMessage.length > 0 || readFileSync(filePath, 'utf-8').includes('5000')).toBe(
true,
);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SHOULD_FIX] This assertion is tautological — it passes whether the gadget throws or succeeds. The test name says "provides suggestions when similar content exists" but doesn't verify that suggestions actually appear in the error.

Consider making this specific to the behavior you want to lock in:

As-is, this test would still pass if the gadget silently returned an empty string or threw a completely unrelated error.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The assertion was indeed tautological — it turned out the original search string (const timeoutMs = 1000;) was close enough to succeed via fuzzy matching, so the gadget never even threw.

I rewrote the test with a search string calibrated to fall between the match threshold (0.8) and suggestion threshold (0.6): it now searches for a structurally similar but sufficiently different function signature. The test now makes three specific assertions:

  1. expect(errorMessage).toMatch(/NOT FOUND/i) — verifies the gadget throws (not silently succeeds)
  2. expect(errorMessage).toMatch(/SIMILAR CONTENT FOUND/i) — verifies suggestions are included
  3. expect(errorMessage).toContain('processOrder') — verifies the actual file content appears in the suggestion

All 18 tests pass, 240/240 unit test files pass.

@zbigniewsobiecki
Copy link
Copy Markdown
Member

@aaight address code review concerns

…tions

Replace the tautological assertion in the "provides suggestions when similar
content exists" test that would pass regardless of gadget behavior. The test
now verifies that the error message contains both "NOT FOUND", "SIMILAR
CONTENT FOUND", and the actual similar content from the file.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@aaight
Copy link
Copy Markdown
Collaborator Author

aaight commented Mar 7, 2026

Request Addressed

What was asked

Address code review concern about a tautological assertion in the "provides suggestions when similar content exists" test.

Changes Made

  • Rewrote the test with a properly calibrated search string that triggers the suggestion engine (similarity between 0.6-0.8) instead of being fuzzy-matched successfully
  • Replaced the tautological errorMessage.length > 0 || file.includes('5000') assertion with three specific assertions:
    • Error contains NOT FOUND (verifies the gadget rejects the search)
    • Error contains SIMILAR CONTENT FOUND (verifies suggestions are generated)
    • Error contains the actual similar content from the file (processOrder)

Verification

  • All 18 tests in fileSearchAndReplace.test.ts passing
  • All 240/240 unit test files (4010 tests) passing
  • Lint and type checks passing (pre-commit hooks)

Copy link
Copy Markdown
Collaborator

@nhopeatall nhopeatall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — Well-structured test file with comprehensive coverage of all FileSearchAndReplace code paths. Follows the established mock pattern from fileInsertContent.test.ts exactly, uses real filesystem operations for end-to-end verification, and all 18 tests pass with CI green.

@zbigniewsobiecki zbigniewsobiecki merged commit e086ed0 into dev Mar 7, 2026
6 checks passed
@zbigniewsobiecki zbigniewsobiecki deleted the feature/file-search-and-replace-tests branch March 16, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants