Add path normalization for consistent cache comparison #112
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This addresses issue #95 where content might not be updated correctly during incremental fetches due to path comparison inconsistencies. Changes: - Add normalizePath() helper that resolves all paths to absolute paths relative to PROJECT_ROOT for consistent comparison - Update hasMissingOutputs() to use normalizePath when checking file existence - Update updatePageInCache() to normalize paths before storage - Fix hasMissingOutputs() to return true for empty outputPaths arrays, ensuring pages with no outputs get regenerated - Add comprehensive test coverage in pathNormalization.test.ts - Update existing tests to expect normalized absolute paths
…ation When merging existing paths with new paths in updatePageInCache, existing paths from older caches (stored in non-normalized format) were being added without normalization. This caused duplicate entries when the same file was represented in different formats. Changes: - Normalize existing paths in updatePageInCache before merging - Add test case verifying old-format paths are properly deduplicated - Comment clarifies this handles migration from older cache formats
…andling Phase 5-6 of issue #95 fix: - Import PROJECT_ROOT in generateBlocks.ts for consistent path resolution - Replace process.cwd() with PROJECT_ROOT for S3 URL path detection - Add integration tests for path normalization edge cases: - Mixed path formats in cache (migration scenario) - Missing outputs detection regardless of format - Path deduplication across updates - Multi-language path consistency
The interface comment incorrectly stated paths were "relative to project root" but they are now stored as absolute paths for consistency. This documentation fix aligns the comment with the actual implementation.
Contributor
🚀 Preview DeploymentYour documentation preview is ready! Preview URL: https://pr-112.comapeo-docs.pages.dev 🔄 Content: Regenerated 5 pages from Notion (script changes detected)
This preview will update automatically when you push new commits to this PR. Built with commit 7090e3b |
Ensures consistent path comparison in the needsProcessing check by normalizing filePath before comparing with cached outputPaths. This prevents edge cases where path format differences could cause incorrect skip/process decisions.
The normalizePath function now correctly distinguishes between: 1. Project-relative paths like /docs/intro.md (where /docs doesn't exist) 2. Genuine system paths like /tmp/foo or /etc/config (where parent exists) Uses filesystem check (fs.existsSync on parent directory) to determine if an absolute path outside PROJECT_ROOT should be preserved as-is or treated as project-relative. Fixes external feedback about /tmp/foo being incorrectly rewritten to PROJECT_ROOT/tmp/foo.
Paths like /a where the parent is "/" should be treated as project-relative, not system paths. The root directory always exists, so we can't use its existence as an indicator of a genuine system path. Added check: if parent directory IS the root (path.parse(parentDir).root === parentDir), don't trust fs.existsSync and treat as project-relative instead. Added tests for: - /a (single directory at root level) - /foo/bar/baz.txt (nested paths where only root exists)
luandro
deleted the
claude/investigate-issue-95-01BL5WJwWJnZx8j68XSQENE8
branch
Contributor
🧹 Preview Deployment CleanupThe preview deployment for this PR has been cleaned up. Preview URL was: Note: Cloudflare Pages deployments follow automatic retention policies. Old previews are cleaned up automatically. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #95
Summary
This PR addresses the issue where content was not being updated correctly during incremental fetches due to path comparison inconsistencies in the page metadata cache.
Problem
The cache stored output paths in inconsistent formats depending on how they were generated:
This caused hasMissingOutputs() and path comparisons to fail when the same file was represented differently, leading to:
Solution
Added a normalizePath() function that resolves all paths to absolute paths relative to PROJECT_ROOT:
Changes
Test Plan
Backward Compatibility