Parallel Refinery: Per-Convoy Refinery Agents

[jrf0110]

Parent

Part of #204 (Phase 4: Hardening)

Problem

The refinery is a per-rig singleton that processes reviews strictly one at a time. When multiple polecats complete work in quick succession, MR beads queue up and wait — each review takes ~2-5 minutes (LLM code review + git merge), so 5 queued reviews take 10-25 minutes of serial processing. processReviewQueue() pops one item per alarm tick, checks if the refinery is idle, and re-queues if it's busy.

This is the single biggest throughput bottleneck in Gastown by Kilo.

Current Architecture

Constraint	Where	Impact
Per-rig singleton	agents.ts:375 — rigSingletonRoles = ['refinery']	Max 1 refinery per rig, ever
One pop per tick	review-queue.ts:197 — LIMIT 1	Even if refinery were parallel, only 1 item dequeued per 5s alarm tick
Busy check bails	Town.do.ts:2786-2791 — re-queues if refinery not idle	Popped item wasted, put back to open
No parallel worktrees	Refinery uses a single worktree per rig	Can't run two reviews on different branches simultaneously

Solution: Per-Convoy Refinery Agents

Introduce a convoy refinery — one refinery agent per active convoy — while keeping the rig-level singleton for standalone beads and final landing merges.

Two refinery tiers

Tier	Scope	Concurrency	Targets
Convoy refinery (new)	One per active convoy	Parallel — multiple convoy refineries can run simultaneously	Intermediate merges: polecat branch → convoy feature branch
Rig singleton refinery (existing)	One per rig	Serial (unchanged)	Standalone beads (non-convoy) and final convoy landing merges (feature branch → main)

Why this works

• Intermediate merges are scoped to a feature branch: Within a convoy, all polecats merge into the convoy's feature branch, not main. Conflicts are localized to that convoy's work. Multiple convoy refineries reviewing different convoys don't interfere with each other at all.
• Main branch access stays serialized: The rig singleton only handles standalone beads and final landing merges. This is inherently lower volume — convoys batch work, so most reviews are intermediate.
• The bottleneck shifts to where it matters: If 5 polecats in a convoy complete work, the convoy refinery can review them in parallel (or rapid succession) instead of queuing behind the singleton for 10-25 minutes.

Implementation

1. Convoy refinery agent lifecycle

• When a convoy transitions to open (or when the first MR bead for a convoy enters the review queue), create a convoy-scoped refinery agent
• The convoy refinery is not a rig singleton — it's a regular agent tied to the convoy bead via a dependency or metadata field
• Multiple convoy refineries can exist per rig simultaneously (one per active convoy)
• The convoy refinery exits when all tracked beads in its convoy have been reviewed (or when the convoy closes/fails)

2. processReviewQueue() changes

• Pop multiple items per alarm tick (up to a configurable limit)
• For each popped item, determine if it belongs to a convoy:
  • Convoy bead: Route to that convoy's refinery (create one if it doesn't exist). If the convoy refinery is busy, re-queue just that item (don't block other convoys or standalone work).
  • Standalone bead: Route to the rig singleton refinery (existing behavior)
  • Convoy landing merge: Route to the rig singleton refinery (existing behavior)
• This means a single alarm tick can dispatch multiple refineries for different convoys

3. Git worktree isolation

Convoy refineries need their own git worktrees, just like polecats:

• Each convoy refinery gets a worktree on the convoy's feature branch
• The rig singleton refinery continues using its existing worktree on the default branch
• agent-runner.ts already supports per-agent worktree creation for polecats — extend to convoy refineries

4. Town config

Add refinery.maxConvoyConcurrency to town config:

• Default: 3 (max 3 convoy refineries running simultaneously per rig)
• This caps total parallel review load on the container
• The rig singleton doesn't count against this limit

5. Refinery system prompt awareness

Convoy refineries get the same prompt as today (with convoyContext.isIntermediateStep = true), but the prompt should note they may be reviewing concurrently with other refineries on the same feature branch. Merge conflicts from concurrent intermediate merges should be resolved normally (the feature branch is a staging area, not production).

Acceptance Criteria

• Convoy-scoped refinery agents created per active convoy (not per-rig singletons)
• processReviewQueue() pops multiple items per alarm tick and routes by convoy vs standalone
• Multiple convoy refineries can run simultaneously across different convoys
• Convoy refineries get their own git worktrees on the convoy's feature branch
• Rig singleton refinery continues to handle standalone beads and final landing merges
• refinery.maxConvoyConcurrency config option with sensible default
• Convoy refinery exits when all tracked beads in its convoy are reviewed
• No regression: standalone (non-convoy) review flow unchanged

Notes

• No data migration needed — cloud Gastown hasn't deployed to production
• The rigSingletonRoles array in agents.ts:375 only needs to stay for the rig-level refinery. Convoy refineries bypass this by being created as regular agents with role refinery and a convoy association in their metadata.
• The review-and-merge convoy mode (each bead merges independently to main) should still use the rig singleton — those beads target the default branch, not a feature branch. Only review-then-land intermediate merges benefit from convoy refineries.
• For review-then-land convoys, if two convoy refineries merge to the feature branch simultaneously and conflict, one will fail the git push. The retry mechanism should handle this — re-queue the failed merge for the next tick with a git pull before retry.

[kilo-code-bot]

This issue appears to be a duplicate of #657.

Bug: Refinery agent dispatched to wrong rig in multi-rig towns (#657)

Similarity score: 90%

This comment was generated by Kilo Auto-Triage.