Skip to content

IssueTracker: GitHub GraphQL rate-limit exceeded — audit and consolidate gh calls #173

@dhilgaertner

Description

@dhilgaertner

Problem

IssueTracker is regularly hitting the GitHub GraphQL rate limit, flooding the console with errors and leaving the ticket/PR UI empty until the reset window. Every major fetch path fails once the quota is blown, so the app silently stops reflecting reality (open issues, PRs, review requests, done-in-last-24h, auto-complete).

Sample log output:

[IssueTracker] fetchGitHubPRs failed: ... GraphQL: API rate limit already exceeded for user ID 1402869.
[IssueTracker] checkSessionPRs: PR lookup for branch 'feature/crow-150-codex-agent-support' failed: ...
[IssueTracker] fetchDoneIssuesLast24h failed: ...
[IssueTracker] checkIssueClosed failed for https://github.com/radiusmethod/crow/issues/150: ...
[IssueTracker] fetchGitHubIssues failed: ...

Hypothesis

IssueTracker.refresh() (Sources/Crow/App/IssueTracker.swift:65) runs every 60s (pollInterval = 60) and fans out into many gh calls, several of which hit the GraphQL search endpoint — which has its own ~30 req/min cap separate from the 5k/hr REST budget:

Per refresh, best case:

  • gh search issues --assignee @me --state open (GraphQL search)
  • gh pr list --author @me --state open (GraphQL)
  • gh search issues --assignee @me --state closed (GraphQL search)
  • gh search prs --review-requested @me + gh pr view per review (GraphQL search + N)
  • fetchGitHubProjectStatuses (GraphQL)

Per-session fan-out (multiplies by number of active sessions):

  • checkSessionPRsgh pr list --repo … --head <branch> per session
  • fetchPRStatuses → N calls
  • autoCompleteFinishedSessionscheckIssueClosed + checkPRMerged per session
  • autoCompleteFinishedReviews → per review

With ~10 active sessions that's 30–50 gh invocations per minute, many going through GraphQL search. Over an hour we easily blow past the search cap and then sit in the error hole until reset.

Investigation / work

  • Instrument refresh() to count gh calls + elapsed time per poll so we have a real baseline
  • Read X-RateLimit-Remaining / X-RateLimit-Reset (gh api exposes them) and expose in AppState for visibility
  • Back off (or skip the next refresh cycle) when remaining < threshold, instead of charging in and failing every call
  • Handle the 403 rate-limit error specifically: parse Retry-After, suspend polling until reset
  • Consolidate: one GraphQL query can return issues + linked PRs + project status in a single round-trip instead of gh search issues + gh pr list + fetchGitHubProjectStatuses
  • Batch checkSessionPRs across all sessions into one query (repo:X head:a head:b head:c) instead of one call per session
  • Drop autoCompleteFinishedSessions/autoCompleteFinishedReviews per-item calls — piggyback on the data already fetched by the main refresh instead of re-querying per session
  • Consider moving from gh CLI to a persistent GraphQL client (reuses auth, lets us batch, avoids process-spawn overhead)
  • Raise pollInterval (e.g. 2–5 min) or switch to event-driven refresh (on window focus, after user action) with a slow background tick

Acceptance

  • Running Crow with ~10 active sessions for an hour produces zero rate-limit errors
  • Refresh cycle issues a bounded, documented number of gh calls regardless of session count (target: O(1) fixed + a small batched O(1) for session PR lookup)
  • Rate-limit exhaustion no longer silently empties the UI — user sees a clear "throttled, retrying at …" state

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions