Skip to content

fix: prevent snapshot from tracking large files#12185

Open
goniz wants to merge 6 commits intoanomalyco:devfrom
goniz:snapshot-filter-bigf
Open

fix: prevent snapshot from tracking large files#12185
goniz wants to merge 6 commits intoanomalyco:devfrom
goniz:snapshot-filter-bigf

Conversation

@goniz
Copy link
Contributor

@goniz goniz commented Feb 4, 2026

This PR prevents the Snapshot.track() flow from adding (untracked) big files into the snapshot system (git).

It changes the code to first do git add -N . which stage the files without content.
Next it iterates over the staged files to find any files larger then defined threshold (1 gig). unstages them AND adds them to $gitdir/info/exclude to prevent them from getting staged again.
Finally it calls git add -- $stagedFiles to stage the files content as well.

To handle existing bloated snapshot dirs, the cleanup method was updated to detect if any existing object blob is bigger then threshold -> nukes the whole snapshot dir.
Leaving the big files causes the git gc --prune process to blow up on memory and hog the system.

Additionally, this PR cleans up the Snapshot code a bit.

Don't be scared from the lines added, I've added tests too 😆

Fixes #6845

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

The following comment was made by an LLM, it may be inaccurate:

Based on the search results, I found one potentially related PR:

Related PR:

You should verify if PR #10628 has already been merged or if it addresses the same root cause as PR #12185.

@goniz goniz force-pushed the snapshot-filter-bigf branch 4 times, most recently from 3f7b472 to 026c6ee Compare February 11, 2026 18:34
@goniz goniz changed the title feat: filter out 1GB+ files from snapshot fix: prevent snapshot from trackibg large files Feb 14, 2026
@goniz goniz changed the title fix: prevent snapshot from trackibg large files fix: prevent snapshot from tracking large files Feb 15, 2026
@goniz goniz force-pushed the snapshot-filter-bigf branch 3 times, most recently from e6cebb3 to 72f72fb Compare February 17, 2026 15:01
@goniz goniz force-pushed the snapshot-filter-bigf branch from 72f72fb to 5bbccec Compare February 17, 2026 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HUGE snapshot folder

1 participant