fix(common): add cleanup for stale file locks to prevent memory leak#44
fix(common): add cleanup for stale file locks to prevent memory leak#44
Conversation
Fixes #5145 and #5142 - File lock HashMaps grow without bound. Problem: Lock entries accumulate forever without cleanup. Solution: Added stale lock cleanup mechanism that removes entries where no external reference exists (strong_count == 1). Cleanup triggers automatically when the map reaches MAX_LOCK_ENTRIES (10,000).
Greptile OverviewGreptile SummaryThis PR addresses memory leaks caused by unbounded growth of file lock HashMaps by implementing automatic cleanup of stale lock entries. Key Changes:
Implementation Approach: Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| src/cortex-common/src/file_locking.rs | Added automatic cleanup of stale lock entries using Arc reference counting, triggered at 10,000 entries |
| src/cortex-resume/src/session_store.rs | Added same stale lock cleanup mechanism to session store's file lock manager |
Sequence Diagram
sequenceDiagram
participant Caller
participant FileLockManager
participant HashMap
participant Arc
Caller->>FileLockManager: get_lock(path)
FileLockManager->>HashMap: lock().unwrap()
HashMap-->>FileLockManager: MutexGuard<HashMap>
alt locks.len() >= MAX_LOCK_ENTRIES (10,000)
FileLockManager->>FileLockManager: cleanup_stale_entries(&mut locks)
loop For each entry in HashMap
FileLockManager->>Arc: strong_count(arc)
Arc-->>FileLockManager: count
alt count == 1
FileLockManager->>HashMap: remove entry (stale)
else count > 1
FileLockManager->>HashMap: keep entry (in use)
end
end
end
FileLockManager->>HashMap: entry(path).or_insert_with(...)
HashMap-->>FileLockManager: Arc<Mutex<()>>
FileLockManager->>Arc: clone()
Arc-->>FileLockManager: Arc<Mutex<()>> (incremented count)
FileLockManager-->>Caller: Arc<Mutex<()>>
Note over Caller,Arc: Caller uses lock for operations
Caller->>Caller: drop Arc reference
Note over Arc: strong_count decrements
alt Last external reference dropped
Note over HashMap: Entry becomes stale (count == 1)
Note over HashMap: Will be cleaned up when threshold reached
end
| fn cleanup_stale_entries( | ||
| locks: &mut std::collections::HashMap<PathBuf, Arc<std::sync::Mutex<()>>>, | ||
| ) { | ||
| locks.retain(|_, arc| Arc::strong_count(arc) > 1); | ||
| } |
There was a problem hiding this comment.
Consider adding a test that creates >10,000 lock entries and verifies cleanup behavior
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/cortex-common/src/file_locking.rs
Line: 604:608
Comment:
Consider adding a test that creates >10,000 lock entries and verifies cleanup behavior
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.This PR consolidates the following memory and storage fixes: - #44: Add cleanup for stale file locks to prevent memory leak - #45: Add cache size limits to prevent unbounded memory growth - #47: Add fsync after file writes to prevent data loss - #50: Bound ToolResponseStore size and cleanup consumed entries - #51: Eliminate TOCTOU races in MCP server and plugin registry - #52: Improve path validation and tilde expansion Key changes: - Added periodic cleanup of stale file locks - Implemented LRU cache limits for config discovery and tokenizer - Added fsync calls after critical file writes - Created bounded ToolResponseStore with automatic cleanup - Fixed time-of-check-time-of-use races - Improved path validation security
|
Consolidated into #80 - fix: consolidated memory and storage improvements |
Summary
Fixes #5145 and #5142 - File lock HashMaps grow without bound.
Problem
Lock entries accumulate forever without cleanup.
Solution
Added stale lock cleanup mechanism to release old entries.