fix: skip CoreML compilation in AsrModels.download()#357
fix: skip CoreML compilation in AsrModels.download()#357Alex-Wengg wants to merge 1 commit intomainfrom
Conversation
AsrModels.download() called DownloadUtils.loadModels() which runs MLModel(contentsOf:configuration:) on all 4 Parakeet models, then discards the loaded objects. That compilation triggers CoreML's internal MPS shader generation which writes gigabytes to disk. On first install the combined download I/O + compilation I/O has been observed to hit ~8.5 GB (writesCaused), causing macOS to kill the process with a disk-write exception (MacBookPro18,1, macOS 15.6.1). Switch download() to use downloadRepo() which fetches files from HuggingFace without loading them into CoreML. Compilation is deferred to load() / downloadAndLoad() where the MLModel objects are actually used. What this does NOT address: the compilation during load() still writes to disk and could still be large. If the crash persists after this change, the load-time compilation itself needs investigation (e.g. compiling models sequentially, or understanding why CoreML generates MPS shaders with .cpuAndNeuralEngine).
|
Claude finished @Alex-Wengg's task —— View job PR Review Complete ✅Todo List:
Review SummaryThis is an excellent fix that properly addresses the critical disk write issue by separating download from CoreML compilation. Key strengths:
Minor considerations: Consider adding lightweight file existence verification after download and ensure progress reporting works correctly with Overall assessment: Well-reasoned solution that should resolve the macOS process kill issue while maintaining compatibility. ✅ |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review Summary
This is an excellent fix that addresses a critical production issue. The change properly separates download concerns from CoreML compilation.
✅ Strengths
- Root cause fix: Correctly identifies and eliminates the unnecessary CoreML compilation during download
- Clear separation of concerns: Download now only fetches files, compilation happens during
load() - Excellent documentation: The comment block clearly explains the problem, impact, and solution
- API compatibility:
downloadAndLoad()method preserves the same behavior for existing callers - Simplified logic: Removes complex per-model handling in favor of straightforward repo download
🔍 Technical Analysis
- Before:
download()→loadModels()→MLModel(contentsOf:)→ immediate discard → 8.5GB disk writes - After:
download()→downloadRepo()→ file fetch only → compilation deferred toload() - Impact: Should dramatically reduce initial disk I/O and prevent macOS process kills
⚠️ Minor Considerations
-
File verification: The previous implementation implicitly verified models could be loaded. Consider adding a lightweight file existence check after download for early error detection.
-
Error handling: Ensure
downloadRepo()provides equivalent error reporting if individual model files fail to download. -
Progress reporting: Verify that progress callbacks work correctly with
downloadRepo()vs the granular per-model progress in the previous implementation.
🧪 Testing Recommendations
- Test first-time installation on clean cache to verify disk write reduction
- Verify
downloadAndLoad()end-to-end functionality - Measure actual disk writes during
download()vsload()phases
Overall: This is a well-reasoned fix that should resolve the immediate production issue while maintaining API compatibility.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Parakeet EOU Benchmark Results ✅Status: Benchmark passed Performance Metrics
Streaming Metrics
Test runtime: 1m41s • 03/07/2026, 01:48 PM EST RTFx = Real-Time Factor (higher is better) • Processing includes: Model inference, audio preprocessing, state management, and file I/O |
PocketTTS Smoke Test ✅
Runtime: 0m25s Note: PocketTTS uses CoreML MLState (macOS 15) KV cache + Mimi streaming state. CI VM lacks physical GPU — audio quality may differ from Apple Silicon. |
Qwen3-ASR int8 Smoke Test ✅
Runtime: 4m7s Note: CI VM lacks physical GPU — CoreML MLState (macOS 15) KV cache produces degraded results on virtualized runners. On Apple Silicon: ~1.3% WER / 2.5x RTFx. |
ASR Benchmark Results ✅Status: All benchmarks passed Parakeet v3 (multilingual)
Parakeet v2 (English-optimized)
Streaming (v3)
Streaming (v2)
Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming 25 files per dataset • Test runtime: 7m21s • 03/07/2026, 01:55 PM EST RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time Expected RTFx Performance on Physical M1 Hardware:• M1 Mac: ~28x (clean), ~25x (other) Testing methodology follows HuggingFace Open ASR Leaderboard |
Offline VBx Pipeline ResultsSpeaker Diarization Performance (VBx Batch Mode)Optimal clustering with Hungarian algorithm for maximum accuracy
Offline VBx Pipeline Timing BreakdownTime spent in each stage of batch diarization
Speaker Diarization Research ComparisonOffline VBx achieves competitive accuracy with batch processing
Pipeline Details:
🎯 Offline VBx Test • AMI Corpus ES2004a • 1049.0s meeting audio • 359.0s processing • Test runtime: 5m 56s • 03/07/2026, 01:55 PM EST |
Speaker Diarization Benchmark ResultsSpeaker Diarization PerformanceEvaluating "who spoke when" detection accuracy
Diarization Pipeline Timing BreakdownTime spent in each stage of speaker diarization
Speaker Diarization Research ComparisonResearch baselines typically achieve 18-30% DER on standard datasets
Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:
🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 39.5s diarization time • Test runtime: 2m 30s • 03/07/2026, 01:55 PM EST |
VAD Benchmark ResultsPerformance Comparison
Dataset Details
✅: Average F1-Score above 70% |
Sortformer High-Latency Benchmark ResultsES2004a Performance (30.4s latency config)
Sortformer High-Latency • ES2004a • Runtime: 2m 47s • 2026-03-07T19:02:41.698Z |
1 participant
Summary
AsrModels.download()calledDownloadUtils.loadModels()which runsMLModel(contentsOf:configuration:)on all 4 Parakeet models, then discards the loaded objects (_ = try await ...). That compilation triggers CoreML's internal MPS shader generation which writes gigabytes to disk.writesCaused), causing macOS to kill the process with a disk-write exception.download()to usedownloadRepo()which fetches files without loading them into CoreML. Compilation is deferred toload()/downloadAndLoad()where theMLModelobjects are actually used.Context
Crash report from a client (MacBookPro18,1, macOS 15.6.1):
What this does NOT fix
The compilation during
load()still writes to disk and could still be large. If the crash persists, the load-time compilation itself needs investigation — e.g. why CoreML generates MPS shaders whencomputeUnitsis.cpuAndNeuralEngine, and whether compiling models one at a time or with.cpuOnlyfor the first load would reduce writes.Test plan
swift buildsucceedsswift test --filter AsrModelsTestspasses (18/18)swift test --filter CITestspasses (13/13)downloadAndLoad()still works end-to-end (downloads + compiles + returns models)download()vsload()separately to confirm reduction🤖 Generated with Claude Code