feat(tui): migrate TUI to in-process app-server

[fcoury]

Problem

The TUI talked to agent threads via a direct CodexThread handle: it created threads through ThreadManager, polled events with next_event(), and forwarded user operations by calling CodexThread::submit(). This by-passed the app-server protocol layer, meaning the TUI could not take advantage of the in-process app-server's session management, config-reload logic, or typed interactive-request routing. It also meant the TUI had no handling for the RequestPermissions tool event, so permission-escalation prompts were silently
dropped.

This is the TUI half of the migration originally planned in #13636. The foundation work (codex-app-server-client facade, in_process runtime host, MessageProcessor extraction) already landed; this PR ports the TUI surface onto it and adds the multi-thread event routing, auth-override nesting, and shared ThreadManager support needed to run child agent sessions in-process. Exec migration is handled separately.

Mental model

There are three layers between the TUI and MessageProcessor:

1. codex_app_server::in_process — low-level runtime host. Owns the MessageProcessor, outbound routing task, and bounded typed channels. Performs the initialize/initialized handshake before returning an InProcessClientHandle.

2. codex-app-server-client — shared facade crate. Wraps the low-level handle behind a worker task with async mpsc channels, adds surface identity (ClientSurface → SessionSource), typed request helpers, backpressure-aware event forwarding, and bounded shutdown with abort fallback.

3. TUI surface (chatwidget/agent.rs) — consumes events from the facade and maps them into the TUI's existing EventMsg / AppEvent vocabulary. No longer performs initialize, session-source selection, or shutdown directly.

After this change the TUI's primary session runs through an in-process InProcessAppServerClient rather than a raw CodexThread. The agent task boots a full app-server runtime, sends thread/start and turn/start JSON-RPC requests, and translates incoming ServerRequest / ServerNotification / legacy-notification events back into EventMsg.

For resumed or forked sessions the TUI still receives a CodexThread from ThreadManager, but registers that thread directly (attach_existing_primary_thread) without spawning a redundant next_event() listener, because the app-server's own event stream already delivers those
events.

A new ApprovalRequest::Permissions variant and MultiSelectPicker widget let the user toggle individual permission grants (network, file-system roots, macOS entitlements) before confirming.

Auth overrides use a stack-based RAII model (ExternalAuthOverrideGuard) so nested in-process sessions can temporarily redirect token refresh without corrupting the shared AuthManager.

Non-goals

• No exec migration — exec is a separate PR.
• No new app-server protocol methods; all existing JSON-RPC methods and events are unchanged.
• No changes to how non-primary (agent-picker) threads are created; they still go through handle_thread_created → register_live_thread.
• The app-server flow is always on for the primary session (active_session_events_via_app_server = true). There is no feature flag to fall back to the old direct-thread path.
• The legacy codex_protocol::Event bridge is preserved; removing it is a separate follow-up.
• No multi-window or split-pane thread display. Only one thread renders at a time; others buffer.

Migration boundaries

Actions kept local in TUI

Handled locally (no app-server RPC needed):

• AddToHistory, GetHistoryEntryRequest, ListCustomPrompts

Forwarded directly to CodexThread (bypassing app-server, functional):

• ReloadUserConfig, OverrideTurnContext, DropMemories, UpdateMemories, RunUserShellCommand, ListMcpTools

Deferred (emit local warning, not functional):

• Undo — emits "temporarily unavailable in in-process local-only mode"

Shutdown is intentionally not a dedicated RPC — it maps to thread/unsubscribe plus local completion handling.

Notifications staying on the legacy event bridge

These notifications are not on the typed app-server path:

• warning, backgroundEvent, turn/streamError
• thread/undo/started, thread/undo/completed, thread/shutdown/completed
• mcpServer/startup/updated, mcpServer/startup/completed
• skills/updated

Three server notifications use the typed app-server path today:

• ServerNotification::ItemStarted (file-change caching)
• ServerNotification::ServerRequestResolved (MCP elicitation cleanup)
• ServerNotification::ThreadClosed (turn-state cleanup, primary-thread shutdown trigger)

Everything else flows through the legacy LegacyNotification → EventMsg bridge, including realtime event consumption (realtime requests were migrated, but event consumption was not).

Other non-migrated pieces

• account/chatgptAuthTokens/refresh remains a local server-request adapter rather than a deeper app-server-owned flow.

Tradeoffs

Decision	Trade-off
Moved ~3700 lines into agent.rs	The agent task now contains the full app-server client loop, protocol adapters, history I/O, and request routing. This makes the file large but keeps the event-loop self-contained.
InProcessAgentContext cloned into every ChatWidget	Avoids lifetime coupling between App and the agent task, at the cost of cloning Arg0DispatchPaths, CLI overrides, and CloudRequirementsLoader on each session creation.
JSON-RPC result envelope preserved on the in-process path	Typed requests still receive serde_json::Value results rather than directly typed structs, because MessageProcessor produces that shape internally. This trades deserialization cost for behavioral parity with socket transports.
Bounded channels with overload rejection	Full queues reject server requests with -32001 (overloaded) instead of blocking or growing unbounded. A slow consumer can cause approval flows to be rejected rather than queued — intentional, since hanging approvals are worse than explicit rejection.
Duplicate-request dedup via same_identity	App-server + legacy event streams can deliver the same approval prompt twice during the migration window. Dedup is applied at enqueue time in ApprovalOverlay, McpServerElicitationOverlay, and RequestUserInputOverlay.
preserve_on_turn_interrupt	MCP elicitation overlays outlive turn interrupts because the MCP server's response expectation is decoupled from the turn lifecycle. Other approval overlays are dismissed.
ClientSurface::Tui maps to SessionSource::Cli	The TUI is the interactive CLI from the server's perspective, so this mapping is correct even though the names look asymmetric.
Two event paths (app-server vs direct)	Fresh sessions use the in-process app-server; fork/resume use ThreadManager + direct listeners. This avoids re-serializing rollout history through app-server but adds routing complexity in App.
Dual approval schemas (V1/V2)	PendingServerRequests tracks both CommandExecutionRequestApproval (V1) and ExecCommandApproval (V2). Preserves backward compatibility during the migration window at the cost of enum duplication.

Architecture

┌──────────────┐        Op         ┌───────────────────────────┐
│  ChatWidget  │ ───────────────►  │  agent task               │
│  (TUI layer) │                   │  InProcessAppServerClient │
│              │ ◄───────────────  │  ┌─ thread/start          │
│              │   AppEvent /      │  ├─ turn/start            │
│              │   EventMsg        │  ├─ approval routing      │
│              │                   │  ├─ history I/O           │
│              │                   │  └─ legacy notification   │
│              │                   │     translation           │
└──────────────┘                   └───────────────────────────┘

Key routing paths:

• spawn_agent starts the in-process client, sends thread/start, and enters the event loop.
• process_in_process_command maps each Op variant to a JSON-RPC client request.
• ServerRequest dispatch maps typed approval requests back to EventMsg events and tracks pending request IDs so responses can be resolved.
• attach_existing_primary_thread registers a thread created by ThreadManager (resume/fork) into the channel infrastructure without starting a competing next_event() listener.

Key type additions

Type	Where	Purpose
ExternalAuthOverrideGuard	core/auth.rs	RAII guard for stack-based auth override nesting; drop removes the override entry
ThreadEventStore	tui/app.rs	Per-thread event buffer with replay-state tracking and snapshot filtering
ThreadEventChannel	tui/app.rs	Sender + receiver + store triple per thread; receiver is taken/returned during thread switches
ThreadScopedOp	tui/chatwidget/agent.rs	Op tagged with target thread_id for cross-thread routing
PendingServerRequests	tui/chatwidget/agent.rs	Request-ID bookkeeping for outstanding interactive approvals, keyed by (thread_id, item_id)
PendingInteractiveReplayState	tui/app/pending_interactive_replay.rs	Set-based tracker for pending prompt call IDs; filters thread-switch snapshots
ApprovalRequest::Permissions	tui/bottom_pane/approval_overlay.rs	New variant for permission-escalation prompts with MultiSelectPicker
ThreadInteractiveRequest	tui/app.rs	Wraps approval/elicitation/user-input from inactive threads for cross-thread bubbling

Contracts

• Terminal event delivery: TurnComplete, TurnAborted, ShutdownComplete use blocking send() on the in-process channel — they are never dropped under backpressure.
• Auth override nesting: push_external_auth_override returns an RAII guard; dropping the guard removes that entry from the stack. A None workspace ID inherits from the nearest outer override.
• Active thread exclusivity: Only one thread's mpsc::Receiver<Event> is active in the select! loop at a time. Switching threads transfers ownership back to ThreadEventChannel.receiver.
• Replay filtering: PendingInteractiveReplayState::should_replay_snapshot_event() suppresses resolved approvals/input when switching threads; TurnComplete/TurnAborted clear all turn-scoped pending state.
• User-input FIFO: request_user_input prompts are answered in arrival order per (thread_id, turn_id). The overlay always resolves the oldest queued RequestId.

Files modified

• codex-rs/tui/src/chatwidget/agent.rs — migrated to use InProcessAppServerClient facade (+3700 lines).
• codex-rs/tui/src/chatwidget.rs — added InProcessAgentContext, RequestPermissions event handling, turn-interrupt overlay cleanup.
• codex-rs/tui/src/app.rs — plumbed Arg0DispatchPaths / CloudRequirementsLoader, added register_live_thread / attach_existing_primary_thread, RequestPermissions approval routing, multi-thread event channel infrastructure.
• codex-rs/tui/src/app/pending_interactive_replay.rs — replay-state tracking for thread-switch snapshot filtering.
• codex-rs/tui/src/bottom_pane/approval_overlay.rs — new Permissions variant, MultiSelectPicker integration, duplicate-request dedup.
• codex-rs/tui/src/bottom_pane/mod.rs — dismiss_turn_scoped_views.
• codex-rs/tui/src/bottom_pane/mcp_server_elicitation.rs — dedup + preserve_on_turn_interrupt.
• codex-rs/tui/src/bottom_pane/request_user_input/mod.rs — dedup, queued multi-request architecture.
• codex-rs/tui/src/chatwidget/interrupts.rs — RequestPermissions variant, thread-scoped interrupt tracking.
• codex-rs/tui/src/bottom_pane/bottom_pane_view.rs — preserve_on_turn_interrupt trait method.
• codex-rs/tui/src/lib.rs — pass arg0_paths and cloud_requirements through to App.
• codex-rs/core/src/auth.rs — stack-based ExternalAuthOverrideGuard with RAII cleanup, workspace inheritance.
• codex-rs/core/src/thread_manager.rs — public auth_manager() accessor, public send_op.
• codex-rs/app-server/src/in_process.rs — in-process runtime with shared ThreadManager support, existing-thread listener attachment.
• codex-rs/app-server/src/message_processor.rs — optional ThreadManager reuse, RAII-based auth override guard.
• codex-rs/app-server-client/src/lib.rs — thread_manager field for shared ThreadManager pass-through.

Observability

• In-process requests emit app_server.request spans with rpc.transport = "in-process", making them distinguishable from stdio/websocket spans in traces.
• Queue saturation warnings are logged via tracing::warn when events are dropped or server requests are rejected due to backpressure.
• send_warning_event / send_error_event surface operational problems as TUI-visible warnings.
• CODEX_TUI_RECORD_SESSION=1 logs inbound AppEvent and outbound Op as timestamped JSONL, including thread_id for multi-thread debugging.
• InProcessServerEvent::Lagged { skipped } propagated to client for transport health monitoring.

Tests

• pending_interactive_replay.rs: 5 new tests covering RequestPermissions lifecycle (enqueue, resolve, turn-abort cleanup, pending-approval flag).
• approval_overlay.rs: 4 new tests — snapshot rendering, granted-subset submission, Ctrl-C deny-all, and duplicate-request dedup.
• mcp_server_elicitation.rs, request_user_input/mod.rs: 1 new dedup test each.
• app.rs: 1 new test proving active_session_events_via_app_server = true suppresses the direct listener.
• chatwidget/tests.rs: existing tests updated for new InProcessAgentContext fields.
• chatwidget/agent.rs: ~1800 lines of test coverage for interactive request lifecycle, V1/V2 approval schemas, turn-scoped cleanup, auth token refresh.
• app-server-client: shared_thread_manager_observes_threads_started_in_process, shared_thread_manager_reuses_configured_auth_manager.
• core/auth.rs: nested_external_auth_override_inherits_lower_workspace_when_unset, override stacking tests.

[Copilot]

Pull request overview

Migrates the TUI (and related CLI agent wiring) to use the new in-process app-server client facade, reducing duplicated “app-server-like” logic in each surface and aligning in-process behavior with the JSON-RPC app-server semantics.

Changes:

• Plumbs Arg0DispatchPaths / CloudRequirementsLoader into the TUI startup path and agent initialization to support in-process app-server startup.
• Replaces the TUI agent thread wiring with an InProcessAppServerClient-backed loop that routes Op ↔ app-server typed requests/events while keeping the legacy event bridge.
• Adds defensive UI behavior to prevent duplicate interactive requests from being queued and to dismiss stale modals on interrupted turns.

Reviewed changes

Copilot reviewed 13 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
codex-rs/tui/src/lib.rs	Threads arg0_paths + cloud_requirements into ratatui app startup.
codex-rs/tui/src/chatwidget.rs	Introduces InProcessAgentContext and updates widget constructors/interrupt handling for the new agent model.
codex-rs/tui/src/chatwidget/agent.rs	New in-process app-server agent loop: request routing, event translation, local history helpers, and bootstrap logic.
codex-rs/tui/src/chatwidget/tests.rs	Updates test init plumbing and adds coverage for reopen/shutdown + interrupted-turn modal behavior.
codex-rs/tui/src/bottom_pane/mod.rs	Adds dismiss_all_views() to clear modal stack on interruption.
codex-rs/tui/src/bottom_pane/request_user_input/mod.rs	Prevents duplicate RequestUserInput requests from being queued twice; adds test.
codex-rs/tui/src/bottom_pane/mcp_server_elicitation.rs	Prevents duplicate MCP elicitation requests from being queued twice; adds test.
codex-rs/tui/src/bottom_pane/approval_overlay.rs	Prevents duplicate approval requests from being queued twice; adds test.
codex-rs/tui/src/app.rs	Stores ambient in-process context and improves handling of existing primary threads during resume/fork flows.
codex-rs/tui/Cargo.toml	Adds dependency on codex-app-server-client.
codex-rs/exec/src/lib.rs	Adds bootstrap helpers for session-configured handling and tightens thread-id validation in server-request handling.
codex-rs/app-server/src/app_server_tracing.rs	Ensures typed in-process spans record a request id via client_request_id().
codex-rs/app-server-client/src/lib.rs	Adds ClientSurface → SessionSource mapping and centralizes overloaded error code constant.
codex-rs/app-server-protocol/src/protocol/common.rs	Minor formatting cleanup.
codex-rs/Cargo.lock	Locks new codex-app-server-client dependency for TUI.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[etraut-openai]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7c78c1f730

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[fcoury]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 040239d412

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[fcoury]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Chef's kiss.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[fcoury]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 481cc32aef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[fcoury]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9ae7d4c1fb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[fcoury]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: af132093ff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".