Skip to content

feat: add performance instrumentation for sync pipeline#215

Open
christopherblaisdell wants to merge 2 commits intodanielcopper:mainfrom
christopherblaisdell:feat/perf-instrumentation-pr
Open

feat: add performance instrumentation for sync pipeline#215
christopherblaisdell wants to merge 2 commits intodanielcopper:mainfrom
christopherblaisdell:feat/perf-instrumentation-pr

Conversation

@christopherblaisdell
Copy link
Copy Markdown

Summary

Add PerfCollector and ETAEstimator to measure sync performance across all pipeline phases. All changes are additive - zero behavioral modifications to existing sync logic.

What's included

  • PerfCollector - tracks per-phase timing, HTTP request counts, bytes transferred, error/retry counts
  • ETAEstimator - provides estimated time remaining during long operations
  • Artwork progress logging - reports downloaded/skipped/failed counts at ~10%% intervals
  • Per-platform ROM fetch timing - individual timing for each platform's API pagination
  • get_perf_report RPC endpoint - exposes collected metrics to the frontend
  • Auto-save perf_report.json - written after each sync completes
  • 35 unit tests for PerfCollector and ETAEstimator

Files changed

  • py_modules/lib/perf.py - NEW: PerfCollector + ETAEstimator
  • py_modules/services/library.py - phase timing wrappers, gauge counters
  • py_modules/adapters/romm/http.py - HTTP request/byte tracking
  • py_modules/services/artwork.py - progress logging
  • py_modules/bootstrap.py - wires PerfCollector to HTTP adapter
  • main.py - get_perf_report RPC endpoint
  • tests/lib/test_perf.py - 35 unit tests
  • pytest.ini - test configuration

Testing

Tested on Steam Deck with 373 ROMs across Dreamcast platform + Metroid collection. Two successful end-to-end syncs confirmed all instrumentation works without affecting sync behavior.

Christopher Blaisdell added 2 commits April 5, 2026 16:38
Add PerfCollector and ETAEstimator to measure sync performance across all phases: per-phase timing, HTTP request counting and byte tracking, per-platform ROM fetch timing, artwork download progress logging at 10 percent intervals, shortcut and stale ROM gauges, get_perf_report RPC endpoint, and auto-save perf_report.json after each sync. Tested on Steam Deck with 373 ROMs. Includes 35 unit tests.
@danielcopper
Copy link
Copy Markdown
Owner

Hey, thanks for the PRs — appreciate the effort!

Before I dive into the individual reviews, I need to understand the shared commit f86bf0d that's in all four PRs. It removes registerGameDetailPatch(), which is the core hook for our entire game detail page (RomMPlaySection, RomMGameInfoPanel, saves tab, achievements, core switching, etc.). That's one of the plugin's main features and it's working fine on the current release.

The code comment mentions a GetAppCountWithToolsFilter TypeError crash in Steam's Library page. Could you share more context on this?

  • What exactly was the crash? (stack trace, console output, or steps to reproduce)
  • Does it happen consistently or only under certain conditions (e.g. during sync, large library, specific SteamOS version)?
  • Was your fork based on the v0.15.0 release tag (ad843fb), or did you have local modifications when you saw this?

I'm asking because I can't merge any of the four PRs as-is since they'd all disable the game detail page. If the crash is real, we'd want to fix the root cause rather than remove the feature. If it was specific to your setup or a transient SteamOS issue, we can just drop that commit.

Once I understand the situation, I'll continue with the detailed review of each PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants