Multi agent orchestrator

ai-temp/AGENTS.md

@@ -0,0 +1,40 @@
+# Multi-Agent Orchestrator Notes
+
+> **Docs in progress** – All multi-agent design material lives in `ai-temp/` while the feature is under active development. **Do not touch files under `docs/` yet;** when the architecture stabilises we will migrate polished sections there.
+
+## Current Capabilities (2025‑10‑20)
+- Delegation runs through the `delegate_agent` tool. Payloads can include `conversation_id` to continue an existing delegate session (`codex-rs/core/src/tools/handlers/delegate.rs`).
+- Discovery/preview/dismiss flows are served by the `delegate_sessions` tool with `list`, `messages`, and `dismiss` operations (`codex-rs/core/src/tools/handlers/delegate_sessions.rs`).
+- `AgentOrchestrator` (`codex-rs/multi-agent/src/orchestrator.rs`) now:
+  - Registers every delegate session, exposing summaries (`DelegateSessionSummary`) and event streams per conversation.
+  - Emits `DelegateEvent::{Started,Delta,Completed,Failed,Info}` via a `SessionEventBroadcaster`.
+  - Supports follow-ups by preserving the original parent run id (`parent_run_for_follow_up`) before re-registering a conversation.
+  - Tracks detached runs and reusable sessions, feeding the `/agent` picker and notifications.
+- TUI integration (`codex-rs/tui/src/app.rs`, `chatwidget.rs`, `history_cell.rs`) provides:
+  - A delegate tree with indentation per depth, status ownership, and summaries.
+  - A picker that offers preview/dismiss actions for saved sessions and detached runs.
+  - Dedicated history cells for preview output (`new_delegate_preview`) and consistent routing so sessions never leak updates into each other.
+- Tests cover the delegate handler, orchestrator follow-up behaviour, and the TUI presentation. See `codex-rs/multi-agent/src/orchestrator/tests.rs` for regression cases on parent linkage and shadow recording.
+
+## Key Modules & Paths
+- Loader & facade: `codex-rs/multi-agent/src/lib.rs` (`AgentConfigLoader`, orchestrator re-export).
+- Runtime: `codex-rs/multi-agent/src/orchestrator.rs`.
+- Shared tool types: `codex-rs/core/src/delegate_tool.rs`.
+- Tool handlers/specs: `codex-rs/core/src/tools/handlers/delegate.rs`, `delegate_sessions.rs`, registry wiring in `codex-rs/core/src/tools/spec.rs`.
+- UI: `codex-rs/tui/src/app.rs`, `app_event.rs`, `chatwidget.rs`, `history_cell.rs`, `/agent` picker, status helpers.
+- Shadow caching architecture: `ai-temp/agents-shadow-client.md`.
+- Follow-up design: `ai-temp/agent-follow-up.md`.
+
+## Behaviour Summary
+1. **New run** – `delegate_agent` validates input and calls `AgentOrchestrator::delegate`. The orchestrator spins up a conversation through `ConversationManager`, registers it, and streams events back to the UI.
+2. **Follow-up** – When `conversation_id` is supplied, `delegate_follow_up` reuses the stored session. Parent run metadata is captured before re-registration so the TUI can keep lineage straight.
+3. **Listing & previews** – `delegate_sessions` pulls from `AgentOrchestrator::list_sessions_paginated` and `recent_messages`, which in turn rely on the shadow cache.
+4. **Detached runs** – `mode: "detached"` returns immediately; runs are tracked until completion and surfaced in the picker with dismiss actions.
+5. **UI routing** – Each event carries `owner_conversation_id`. `App::handle_delegate_update` forwards deltas and completions only to the matching session handle, while parent summaries are enqueued via `ChildSummary`.
+
+## Outstanding / Planned
+- Agent switching (temporarily entering a delegate session) remains a future task – see `ai-temp/agent-switching.md`.
+- No edits to `docs/advanced.md` (rolled back) or other public docs until this feature graduates.
+- Additional end-to-end CLI tests and documentation polish still required before release.
+
+For subsystem-specific details, consult the dedicated docs in `ai-temp/` (shadow client, error handling, parallel delegation, persistence, follow-ups, etc.). Each file references the relevant source paths so updates stay aligned with implementation.

ai-temp/CHANGELOG.md

@@ -0,0 +1,44 @@
+# Multi-Agent Orchestrator Change Log
+
+> Keep this file current; it documents the evolution of the multi-agent design work. An outdated changelog breaks the orchestrator timeline.
+
+## 2025-10-27
+
+- Added follow-up delegation support: orchestrator can resume saved sessions, shadow-backed previews power the TUI picker, and the `delegate_sessions` tool exposes list/messages/dismiss operations.
+- Documented the shipped behaviour inside `ai-temp/` (AGENTS overview, orchestration integration, UI/UX notes); public docs remain untouched until release.
+- Implemented dismissal plumbing so stale sessions and their shadow caches can be removed safely from both the CLI and coordinating prompts.
+
+## 2025-10-19
+
+- Reworked delegate event fan-in so each conversation feeds a dedicated `spawn_event_forwarder` and surfaces updates through `AppEvent::CodexEvent { conversation_id, event }`, keeping history rendering scoped to the active session.
+- Added shadow-session plumbing to the orchestrator (`ShadowManager`, per-session metrics, user/agent capture forwarding) and exposed the new telemetry through the TUI status panel.
+- Expanded the delegate app events (`DelegateShadow*`, `InsertHistoryCell { conversation_id, ... }`) and updated the TUI’s `SessionHandle` storage to hydrate from snapshots instead of replaying global history.
+- Refreshed `ai-temp/agents-shadow-client.md`, `ai-temp/orchestration-integration.md`, and the sample Codex home (`example-codex-home/AGENTS.md`) to describe per-session streams, `DelegateEvent::{Started,Delta,Completed,Failed}`, and the tightened coordinator instructions.
+
+## 2025-10-16
+
+- Added batched delegate execution: the core handler now accepts `batch` payloads, launches every child run concurrently (respecting the orchestrator’s concurrency cap), and returns per-agent summaries in a single response.
+- Updated the TUI delegate tree to indent siblings (two spaces per depth) and keep the status banner aligned while multiple children stream at once; adjusted snapshot/unit coverage accordingly.
+- Refreshed documentation and sample instructions (`ai-temp/parallel-delegation.md`, `ai-temp/tool-implementation-patterns.md`, example Codex home agents) to describe the batched call flow and new presentation.
+- Removed the inline `#agent` autocomplete experiment and reverted documentation to focus on the delegate picker and slash command flow.
+- Added child delegate directories (`creative_ideas`, `conservative_ideas`) to the example Codex home and updated instructions/README to describe the fixed delegation chain (main → ideas_provider → critic).
+- Auto-enabled the delegate tool whenever `[multi_agent].agents` is non-empty so sub-agents inherit delegation without toggling `include_delegate_tool`.
+- Updated `AgentOrchestrator` to spawn sub-agent conversations via `ConversationManager::with_delegate`, enabling delegates to invoke their own delegates.
+- Switched delegate execution tracking to a stack; the TUI now shows nested runs with indented history lines.
+- Added focused unit tests covering the new config flag behaviour and UI indentation to prevent regressions.
+
+## 2025-10-14
+
+- Captured the baseline design artifacts (`AGENTS.md`, `config-design.md`, `instruction-design.md`, `persistence-design.md`, `error-handling.md`) compiled during the planning phase.
+- Reiterated the requirement that this changelog must stay up to date as the multi-agent feature evolves.
+- Scaffolded the `codex-multi-agent` crate with `AgentId`, `AgentRegistry`, and async config loading that merges global/agent/CLI overrides into an `AgentContext`.
+- Wired the TUI bootstrapper to the new loader, introducing a `--agent` flag that scopes interactive runs to `~/.codex/agents/<agent_id>/`.
+- Added `ai-temp/example-codex-home/` with ready-to-run config, instructions, and multiple agent directories for hands-on testing via `CODEX_HOME=...` and `--agent`.
+- Authored `ai-temp/orchestration-integration.md`, outlining logic, UI/UX, and minimal-coupling hooks to let the primary agent delegate work to sub-agents in the existing codebase.
+- Captured delegation decisions (single-flight execution, shared auth, primary-agent-composed prompts) inside `ai-temp/orchestration-integration.md`.
+- Simplified the example Codex home to `ideas_provider` (gpt-5) and `critic` (gpt-5-nano) agents for easier manual testing.
+- Delegated runs now stream live output (`DelegateEvent::Delta`) through the TUI, and remaining UX follow-ups are tracked in `ai-temp/ui-ux-delegation.md`.
+- Added a dedicated status indicator while a delegate runs, restored the idle header on completion, and regression-tested streaming to prevent animation regressions.
+- Updated the sample Codex home instructions/README, ensured the critic agent uses `gpt-5-nano`, and documented the new delegation UX in `ai-temp/ui-ux-delegation.md`.
+- Documented plan-tool implementation patterns and how they inform future delegation tools (`ai-temp/tool-implementation-patterns.md`).
+- Observed that the coordinator remains silent between delegate runs (e.g., between `#ideas_provider` completion and the `#critic` request) because tool composition happens model-side without emitting UI events; leaving this behaviour in place for now.

ai-temp/TODO.md

@@ -0,0 +1,25 @@
+# Follow-Up Rendering Bug (UI)
+
+We found that delegate follow-up prompts disappear when reopening the saved session even though the
+run’s rollout contains them. Root cause:
+
+- `App::activate_delegate_session` only calls `ChatWidget::hydrate_from_shadow` when
+  `SessionHandle::history().is_empty()`.
+- On follow-up, the handle still has prior history, so hydration is skipped and the old transcript
+  remains.
+
+## Next Steps
+1. Adjust `activate_delegate_session` to refresh from the shadow snapshot even when history already
+   exists (e.g., clear/replace history or always hydrate).
+2. Add a regression test that currently fails: create a `SessionHandle` with non-empty history, mock
+   an `ActiveDelegateSession` snapshot containing follow-up user and agent messages, invoke
+   `activate_delegate_session`, and assert the transcript now includes the follow-up prompt.
+3. After the fix, run `cargo test -p codex-tui` to confirm coverage.
+
+## Useful References
+- Rollout file with the missing follow-up:
+  `ai-temp/example-codex-home/agents/critic/sessions/2025/10/21/rollout-2025-10-21T00-33-16-019a042f-3e55-7171-b9df-1690b9f905a0.jsonl`
+- Entry points:
+  - `codex-rs/tui/src/app.rs::activate_delegate_session`
+  - `codex-rs/tui/src/chatwidget.rs::hydrate_from_shadow`
+  - Shadow capture logic in `codex-rs/multi-agent/src/shadow/{mod,recorder}.rs`