Persist text element ranges and attached images across history/resume by charley-oai · Pull Request #9116 · openai/codex

charley-oai · 2026-01-12T22:59:27Z

Summary

Backtrack selection now rehydrates text_elements and local_image_paths from the chosen user history cell so Esc‑Esc history edits preserve image placeholders and attachments.
Composer prefill uses the preserved elements/attachments in both tui and tui2.
Extended backtrack selection tests to cover image placeholder elements and local image paths.

Changes

tui/src/app_backtrack.rs: Backtrack selection now carries text elements + local image paths; composer prefill uses them (removes TODO).
tui2/src/app_backtrack.rs: Same as above.
tui/src/app.rs: Updated backtrack test to assert restored elements/paths.
tui2/src/app.rs: Same test updates.

The original scope of this PR (threading text elements and image attachments through the codex harness thoroughly/persistently) was broken into the following PRs other than this one:

The diff of this PR was reduced by changing types in a starter PR: #9235

Then text element metadata was added to protocol, app server, and core in this PR: #9331

Then the end-to-end flow was completed by wiring TUI/TUI2 input, history, and restore behavior in #9393

Prompt expansion was supported in this PR: #9518

TextElement optional placeholder field was protected in #9545

Copilot

Pull request overview

This PR adds support for persisting text element ranges and attached images across history, resume, and backtracking operations. Text elements are byte-span metadata that enable the UI to highlight special content (e.g., image placeholders) in user messages without modifying the underlying text. The changes ensure that when users edit history or backtrack, the UI can restore both the original text and its associated visual markers, including rehydrating local image attachments.

Changes:

Added text_elements and local_images fields to user input/message events with backward-compatible serialization defaults
Threaded text elements through protocol, app-server, core, and TUI layers to preserve UI metadata across history operations
Refactored backtracking flow to be asynchronous, requesting conversation history before forking threads and properly rehydrating composer state

Reviewed changes

Copilot reviewed 89 out of 89 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
`codex-rs/protocol/src/user_input.rs`	Added `TextElement` and `ByteRange` types to represent UI spans within text
`codex-rs/protocol/src/protocol.rs`	Added `local_images` and `text_elements` fields to `UserMessageEvent`
`codex-rs/protocol/src/items.rs`	Added methods to extract and rebase text elements from user message content
`codex-rs/tui2/src/bottom_pane/textarea.rs`	Added support for setting/getting text with element ranges
`codex-rs/tui2/src/bottom_pane/chat_composer.rs`	Enhanced submission logic to capture, trim, and preserve text elements through the input pipeline
`codex-rs/tui2/src/history_cell.rs`	Added rendering logic to display text elements with cyan styling in history
`codex-rs/tui2/src/app_backtrack.rs`	Refactored backtrack to async two-step flow with conversation history request and proper error handling
`codex-rs/tui/src/*`	Mirror of TUI2 changes for the original TUI implementation
`codex-rs/core/src/codex.rs`	Modified user turn handling to emit events from `UserInput` to preserve UI metadata
`codex-rs/app-server/*`	Propagated text elements through v1/v2 API types and history rebuilding
Test files	Updated tests to include empty `text_elements` and `local_images` vectors

Comments suppressed due to low confidence (2)

codex-rs/tui2/src/bottom_pane/chat_composer.rs:1

The Enter key handling now queues messages when steer_enabled is false, which changes the previous behavior where Enter always submitted immediately. This logic should match line 1046 in the test file where Tab was changed to Enter, suggesting the original intent was for Enter to always submit. The should_queue logic appears inverted or the comment at the old line 1615 ("Enter always sends messages immediately") is now incorrect.
codex-rs/tui2/src/app_backtrack.rs:1
The struct fields text, text_elements, and local_image_paths in BacktrackPrefill should be private (not pub) since the struct is marked pub(crate). Public fields on a crate-internal struct break encapsulation and may allow incorrect mutations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 89 out of 89 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codex-rs/app-server-protocol/src/protocol/v2.rs

Initial type tweaking PR to make the diff of #9116 smaller This should not change any behavior, just adds some fields to types

aibrahim-oai · 2026-01-15T07:28:29Z

@codex review this

codex-rs/app-server-protocol/src/protocol/v1.rs

codex-rs/core/src/codex.rs

codex-rs/protocol/src/items.rs

aibrahim-oai

I feel the PR is still pretty big and has a lot of moving parts.

I can see it broken down to support in app-server, core, and maybe two-stacked for tui one for general support of the text element and one for the backtracking changes

That way it be easier to track those moving parts and remove unnecessary changes. Also, will enable us to think carefully about how to test it.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 600f1a15bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

codex-rs/tui2/src/chatwidget.rs

codex-rs/tui2/src/bottom_pane/chat_composer.rs

The second part of breaking up PR #9116 Summary: - Add `TextElement` / `ByteRange` to protocol user inputs and user message events with defaults. - Thread `text_elements` through app-server v1/v2 request handling and history rebuild. - Preserve UI metadata only in user input/events (not `ContentItem`) while keeping local image attachments in user events for rehydration. Details: - Protocol: `UserInput::Text` carries `text_elements`; `UserMessageEvent` carries `text_elements` + `local_images`. Serialization includes empty vectors for backward compatibility. - app-server-protocol: v1 defines `V1TextElement` / `V1ByteRange` in camelCase with conversions; v2 uses its own camelCase wrapper. - app-server: v1/v2 input mapping includes `text_elements`; thread history rebuilds include them. - Core: user event emission preserves UI metadata while model history stays clean; history replay round-trips the metadata.

codex-rs/tui/src/app.rs

Continuation of breaking up this PR #9116 ## Summary - Thread user text element ranges through TUI/TUI2 input, submission, queueing, and history so placeholders survive resume/edit flows. - Preserve local image attachments alongside text elements and rehydrate placeholders when restoring drafts. - Keep model-facing content shapes clean by attaching UI metadata only to user input/events (no API content changes). ## Key Changes - TUI/TUI2 composer now captures text element ranges, trims them with text edits, and restores them when submission is suppressed. - User history cells render styled spans for text elements and keep local image paths for future rehydration. - Initial chat widget bootstraps accept empty `initial_text_elements` to keep initialization uniform. - Protocol/core helpers updated to tolerate the new InputText field shape without changing payloads sent to the API.

charley-oai · 2026-01-21T02:41:43Z

@codex review this

chatgpt-codex-connector · 2026-01-21T02:43:58Z

Codex Review: Didn't find any major issues. 👍

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4808391ff1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

codex-rs/tui/src/app_backtrack.rs

charley-oai · 2026-01-21T03:23:46Z

@codex review this

chatgpt-codex-connector · 2026-01-21T03:34:05Z

Codex Review: Didn't find any major issues. Can't wait for the next one!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

charley-oai · 2026-01-21T20:14:06Z

codex-rs/tui/src/bottom_pane/chat_composer.rs

        }

+        // Keep image placeholders normalized to [Image #1].. in attachment order.
+        self.relabel_attached_images_and_update_placeholders();


Codex pointed out that the external edits path didn't relabel attached images, which could break the image labeling assumptions made by this PR

charley-oai · 2026-01-22T18:03:53Z

@codex review this

chatgpt-codex-connector · 2026-01-22T18:10:14Z

Codex Review: Didn't find any major issues. More of your lovely PRs please.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

charley-oai changed the title ~~Persist text element ranges across history/resume~~ Persist text element ranges and attached images across history/resume Jan 13, 2026

charley-oai force-pushed the save-text-elements-in-user-input branch from 5f1939c to 1affb03 Compare January 14, 2026 01:29

etraut-openai added the oai PRs contributed by OpenAI employees label Jan 14, 2026

charley-oai force-pushed the save-text-elements-in-user-input branch from b04477d to 93d0571 Compare January 14, 2026 05:36

charley-oai requested a review from Copilot January 14, 2026 05:39

Copilot AI reviewed Jan 14, 2026

View reviewed changes

charley-oai requested a review from Copilot January 14, 2026 05:55

Copilot AI reviewed Jan 14, 2026

View reviewed changes

owenlin0 reviewed Jan 14, 2026

View reviewed changes

codex-rs/app-server-protocol/src/protocol/v2.rs Outdated Show resolved Hide resolved

owenlin0 reviewed Jan 14, 2026

View reviewed changes

codex-rs/app-server-protocol/src/protocol/v2.rs Show resolved Hide resolved

charley-oai force-pushed the save-text-elements-in-user-input branch from 2cb95a9 to 2a47905 Compare January 14, 2026 21:28

charley-oai mentioned this pull request Jan 14, 2026

Add text element metadata to types #9235

Merged

charley-oai added a commit that referenced this pull request Jan 15, 2026

Add text element metadata to types (#9235)

4a9c2bc

Initial type tweaking PR to make the diff of #9116 smaller This should not change any behavior, just adds some fields to types

charley-oai force-pushed the save-text-elements-in-user-input branch 3 times, most recently from d01cf2b to f098faf Compare January 15, 2026 04:52

aibrahim-oai reviewed Jan 15, 2026

View reviewed changes

codex-rs/app-server-protocol/src/protocol/v1.rs Show resolved Hide resolved

aibrahim-oai reviewed Jan 15, 2026

View reviewed changes

codex-rs/core/src/codex.rs Outdated Show resolved Hide resolved

aibrahim-oai reviewed Jan 15, 2026

View reviewed changes

codex-rs/protocol/src/items.rs Show resolved Hide resolved

aibrahim-oai reviewed Jan 15, 2026

View reviewed changes

chatgpt-codex-connector bot reviewed Jan 15, 2026

View reviewed changes

codex-rs/tui2/src/chatwidget.rs Outdated Show resolved Hide resolved

codex-rs/tui2/src/bottom_pane/chat_composer.rs Outdated Show resolved Hide resolved

charley-oai mentioned this pull request Jan 15, 2026

Add text element metadata to protocol, app server, and core #9331

Merged

charley-oai force-pushed the save-text-elements-in-user-input branch from 28350b4 to 7625377 Compare January 16, 2026 18:58

This was referenced Jan 16, 2026

Persist text elements through TUI input and history #9390

Closed

Persist text elements through TUI input and history #9393

Merged

charley-oai force-pushed the save-text-elements-in-user-input branch from a641172 to a23e6d6 Compare January 18, 2026 04:32

charley-oai changed the base branch from main to tui-text-elements-history January 18, 2026 04:33

charley-oai commented Jan 19, 2026

View reviewed changes

codex-rs/tui/src/app.rs Show resolved Hide resolved

charley-oai force-pushed the tui-text-elements-history branch 2 times, most recently from 1329889 to f96ab1f Compare January 20, 2026 06:39

Base automatically changed from tui-text-elements-history to main January 20, 2026 07:49

charley-oai force-pushed the save-text-elements-in-user-input branch 2 times, most recently from d1044cf to 5e35681 Compare January 20, 2026 18:53

charley-oai marked this pull request as draft January 20, 2026 22:09

charley-oai force-pushed the save-text-elements-in-user-input branch 4 times, most recently from 58deaa1 to 4808391 Compare January 21, 2026 02:41

chatgpt-codex-connector bot reviewed Jan 21, 2026

View reviewed changes

codex-rs/tui/src/app_backtrack.rs Show resolved Hide resolved

charley-oai commented Jan 21, 2026

View reviewed changes

charley-oai marked this pull request as ready for review January 21, 2026 20:17

charley-oai added 2 commits January 22, 2026 10:03

Support text elements and attached images in backtracking / esc esc

1020ca4

Fix bug

aebb2f3

charley-oai force-pushed the save-text-elements-in-user-input branch from b3aa0f2 to aebb2f3 Compare January 22, 2026 18:03

aibrahim-oai approved these changes Jan 23, 2026

View reviewed changes

charley-oai merged commit 935d88b into main Jan 23, 2026
36 of 38 checks passed

charley-oai deleted the save-text-elements-in-user-input branch January 23, 2026 18:18

github-actions bot locked and limited conversation to collaborators Jan 23, 2026

Conversation

charley-oai commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The original scope of this PR (threading text elements and image attachments through the codex harness thoroughly/persistently) was broken into the following PRs other than this one:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

aibrahim-oai commented Jan 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aibrahim-oai left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

charley-oai commented Jan 21, 2026

Uh oh!

chatgpt-codex-connector bot commented Jan 21, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

charley-oai commented Jan 21, 2026

Uh oh!

chatgpt-codex-connector bot commented Jan 21, 2026

Uh oh!

charley-oai Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

charley-oai commented Jan 22, 2026

Uh oh!

chatgpt-codex-connector bot commented Jan 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

charley-oai commented Jan 12, 2026 •

edited

Loading