fix: resolve WinError 32 during sparse-checkout fallback on Windows#235
Conversation
4b3e5b1 to
da920e7
Compare
There was a problem hiding this comment.
Pull request overview
Fixes a Windows-specific installation failure (WinError 32) when downloading subdirectory packages via sparse-checkout and its fallback paths by ensuring Git/GitPython resources are closed before cleanup and by making cleanup more robust.
Changes:
- Add Windows-safe cleanup helpers (
_close_repo,_rmtree) and switch subdirectory download temp handling fromTemporaryDirectorytomkdtemp+try/finally. - Change full-clone fallback to use a fresh
repo_clone/directory instead of reusing the sparse-checkout path. - Add tests covering the new cleanup helpers and the updated sparse-checkout/fallback behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
src/apm_cli/deps/github_downloader.py |
Introduces _close_repo/_rmtree and updates download_subdirectory_package temp clone/cleanup flow to avoid Windows file-handle races. |
tests/test_github_downloader.py |
Adds unit tests for cleanup helpers and for sparse-checkout success/failure cleanup behavior. |
| repo = MagicMock() | ||
| repo.git.clear_cache.side_effect = RuntimeError("git gone") | ||
| # Must not propagate | ||
| _close_repo(repo) |
There was a problem hiding this comment.
test_close_repo_swallows_exceptions() only asserts that _close_repo() doesn’t raise when repo.git.clear_cache() fails, but it doesn’t verify that repo.close() is still invoked. Since the main goal is to release handles, the test should assert that close() is called even if clear_cache() throws (this will also catch regressions where _close_repo stops short of closing on errors).
| _close_repo(repo) | |
| _close_repo(repo) | |
| # Even if clear_cache fails, we must still attempt it and close the repo | |
| repo.git.clear_cache.assert_called_once() | |
| repo.close.assert_called_once() |
There was a problem hiding this comment.
Hey @JanDeDobbeleer — welcome to APM and thank you for this fantastic first contribution! 🎉
The root cause analysis in the PR description is top-notch — clearly identifying the two distinct failure paths (sparse-checkout fallback vs. success) and the GitPython handle lifecycle issue. The pre/post screenshot is a great touch too. This is exactly the kind of thorough, well-documented bug fix we love to see.
The approach is clean and well-scoped: mkdtemp + try/finally for explicit cleanup control, fresh repo_clone/ directory to sidestep locked paths, and _close_repo() to release handles eagerly. Tests cover all the right scenarios. CI is all green. ✅
I have just two small suggestions on Copilot's review above to tighten the robustness of the new helpers — both are quick fixes. Once those are in, this is ready to merge. Really appreciate you taking the time to track this down and fix it properly!
When sparse-checkout fails (unsupported git version, network error, auth), the partial .git directory was removed with a plain shutil.rmtree. On Windows, git subprocesses (git-remote-https.exe and background maintenance) can briefly outlive their parent and hold handles on index/pack files, causing: [WinError 32] The process cannot access the file because it is being used by another process: '...\\repo' Fix: add _rmtree() helper that uses shutil.rmtree's onerror callback to make read-only git files writable, with a single 0.5s retry for brief Windows handle races. Use it for the sparse-checkout fallback cleanup in download_subdirectory_package. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
da920e7 to
18a48b7
Compare
|
@danielmeppiel done |
Description
Problem
On Windows,
apm installconsistently failed with:❌ Failed to install github.com/.../skills/conventional-commit: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users...\AppData\Local\Temp\tmpXXXXX\repo'
All packages installed via sparse checkout (subdirectory packages like skills) were affected.
Root cause
Two distinct failure paths, both caused by git file handles still being open when cleanup ran:
Path A — 20% progress (sparse checkout fails, falls back to full clone) When
_try_sparse_checkoutfailed, the code tried toshutil.rmtreethe partial sparse-checkout directory before cloning into the same path. On Windows,git fetchspawnsgit-remote-https.exeas a child process whose handles linger asynchronously aftersubprocess.runreturns. The rmtree would fail silently (locked files remain), thenRepo.clone_frominto the non-empty directory would fail.TemporaryDirectory.__exit__then called plainshutil.rmtreeagain — raising WinError 32 as the visible error.Path B — 70% progress (sparse checkout succeeds) After extracting the subdirectory, a
Repo(temp_clone_path)object was created to read the commit SHA. This GitPython object held.git/indexhandles open. When thewith tempfile.TemporaryDirectory()block exited, its__exit__calledshutil.rmtreewithout any retry logic, hitting WinError 32 on those open handles.Fix
Three targeted changes in
download_subdirectory_package:TemporaryDirectory→mkdtemp+try/finally+_rmtree— gives explicit control over cleanup timing and uses the retry-capable_rmtreehelper instead of the plainshutil.rmtreeinsideTemporaryDirectory.__exit__.Full-clone fallback uses a fresh
repo_clone/subdirectory instead of re-using the (possibly locked)repo/sparse-checkout path. Thelocked directory is left untouched until
_rmtree(temp_dir)in thefinallyblock, by which point all git subprocesses have long exited.Added
_close_repo()helper that callsrepo.git.clear_cache()+repo.close()to explicitly release GitPython file handles. Applied tothe commit-SHA capture
Repoobject so handles are freed before_rmtreeruns.Tests
Added
TestWindowsCleanupHelpers(5 tests) andTestDownloadSubdirectoryPackageWindowsCleanup(2 tests) covering:_rmtreeremoves normal and read-only directories_close_repohandlesNone, releases handles, and swallows exceptionsRepois closed before_rmtreein the success pathrepo_clone/, never the sparse pathPre-Post screenshot
Type of change
Testing