feat: add performance instrumentation for sync pipeline#215
feat: add performance instrumentation for sync pipeline#215christopherblaisdell wants to merge 2 commits intodanielcopper:mainfrom
Conversation
Add PerfCollector and ETAEstimator to measure sync performance across all phases: per-phase timing, HTTP request counting and byte tracking, per-platform ROM fetch timing, artwork download progress logging at 10 percent intervals, shortcut and stale ROM gauges, get_perf_report RPC endpoint, and auto-save perf_report.json after each sync. Tested on Steam Deck with 373 ROMs. Includes 35 unit tests.
|
Hey, thanks for the PRs — appreciate the effort! Before I dive into the individual reviews, I need to understand the shared commit The code comment mentions a
I'm asking because I can't merge any of the four PRs as-is since they'd all disable the game detail page. If the crash is real, we'd want to fix the root cause rather than remove the feature. If it was specific to your setup or a transient SteamOS issue, we can just drop that commit. Once I understand the situation, I'll continue with the detailed review of each PR. |
Summary
Add PerfCollector and ETAEstimator to measure sync performance across all pipeline phases. All changes are additive - zero behavioral modifications to existing sync logic.
What's included
Files changed
py_modules/lib/perf.py- NEW: PerfCollector + ETAEstimatorpy_modules/services/library.py- phase timing wrappers, gauge counterspy_modules/adapters/romm/http.py- HTTP request/byte trackingpy_modules/services/artwork.py- progress loggingpy_modules/bootstrap.py- wires PerfCollector to HTTP adaptermain.py- get_perf_report RPC endpointtests/lib/test_perf.py- 35 unit testspytest.ini- test configurationTesting
Tested on Steam Deck with 373 ROMs across Dreamcast platform + Metroid collection. Two successful end-to-end syncs confirmed all instrumentation works without affecting sync behavior.