Skip to content

⚡ Bolt: parallelize NodeModulesScanner #211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
acebytes wants to merge 1 commit into from
+11 −8
Closed

⚡ Bolt: parallelize NodeModulesScanner #211

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .jules/bolt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
## 2026-05-04 - Fix Actor Serialization in TaskGroup
**Learning:** When an actor creates a `TaskGroup` and its subtasks call an `isolated` method on the same actor, execution is silently serialized on the actor's executor, destroying intended parallelism. Additionally, separate `FileManager.fileExists` and `attributesOfItem` calls create redundant syscalls.
**Action:** Mark computationally intensive or state-independent methods as `nonisolated` so they run on the global concurrent pool. Use `URL.resourceValues(forKeys:)` for combined file existence and attribute checking in a single syscall.
16 changes: 8 additions & 8 deletions Sources/Cacheout/Scanner/NodeModulesScanner.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ actor NodeModulesScanner {
let rootURL = home.appendingPathComponent(root)
guard fileManager.fileExists(atPath: rootURL.path) else { continue }
group.addTask {
await self.findNodeModules(in: rootURL, maxDepth: maxDepth)
await self.findNodeModules(in: rootURL, fileManager: self.fileManager, maxDepth: maxDepth)
}
}
for await items in group {
Expand All @@ -77,18 +77,18 @@ actor NodeModulesScanner {
.sorted { $0.sizeBytes > $1.sizeBytes }
}

private func findNodeModules(in directory: URL, maxDepth: Int, currentDepth: Int = 0) async -> [NodeModulesItem] {
nonisolated private func findNodeModules(in directory: URL, fileManager: FileManager, maxDepth: Int, currentDepth: Int = 0) async -> [NodeModulesItem] {
guard currentDepth < maxDepth else { return [] }

var results: [NodeModulesItem] = []
let nodeModulesURL = directory.appendingPathComponent("node_modules")

// Check if this directory contains node_modules
var isDir: ObjCBool = false
if fileManager.fileExists(atPath: nodeModulesURL.path, isDirectory: &isDir), isDir.boolValue {
let size = directorySize(at: nodeModulesURL)
if let values = try? nodeModulesURL.resourceValues(forKeys: [.isDirectoryKey, .contentModificationDateKey]),
values.isDirectory == true {
Comment on lines +87 to +88
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot May 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle symlinked node_modules directories

This check regresses detection of node_modules when the path is a symbolic link to a directory. URL.resourceValues(...).isDirectory reports false for symlinks, while the previous fileExists(atPath:isDirectory:) path treated a symlinked directory as a directory, so these projects are now silently omitted from scan results. This affects users who relocate node_modules via symlink (for example to another volume) and leads to under-reporting reclaimable disk usage.

Useful? React with 👍 / 👎.

let size = directorySize(at: nodeModulesURL, fileManager: fileManager)
if size > 0 {
let lastMod = try? fileManager.attributesOfItem(atPath: nodeModulesURL.path)[.modificationDate] as? Date
let lastMod = values.contentModificationDate
let projectName = directory.lastPathComponent
results.append(NodeModulesItem(
projectName: projectName,
Expand All @@ -113,14 +113,14 @@ actor NodeModulesScanner {
let name = item.lastPathComponent
guard !Self.skipDirs.contains(name) else { continue }
guard (try? item.resourceValues(forKeys: [.isDirectoryKey]).isDirectory) == true else { continue }
let subResults = await findNodeModules(in: item, maxDepth: maxDepth, currentDepth: currentDepth + 1)
let subResults = await findNodeModules(in: item, fileManager: fileManager, maxDepth: maxDepth, currentDepth: currentDepth + 1)
results.append(contentsOf: subResults)
}

return results
}

private func directorySize(at url: URL) -> Int64 {
nonisolated private func directorySize(at url: URL, fileManager: FileManager) -> Int64 {
var total: Int64 = 0
guard let enumerator = fileManager.enumerator(
at: url,
Expand Down