Skip to content

Align SQLite feedback logs with feedback formatter#13494

Merged
charley-oai merged 11 commits intomainfrom
cc/feedback-log-body-rename
Mar 18, 2026
Merged

Align SQLite feedback logs with feedback formatter#13494
charley-oai merged 11 commits intomainfrom
cc/feedback-log-body-rename

Conversation

@charley-oai
Copy link
Collaborator

@charley-oai charley-oai commented Mar 4, 2026

Summary

  • store a pre-rendered feedback_log_body in SQLite so /feedback exports keep span prefixes and structured event fields
  • render SQLite feedback exports with timestamps and level prefixes to match the old in-memory feedback formatter, while preserving existing trailing newlines
  • count feedback_log_body in the SQLite retention budget so structured or span-prefixed rows still prune correctly
  • bound /feedback row loading in SQL with the retention estimate, then apply exact whole-line truncation in Rust so uploads stay capped without splitting lines

Details

  • add a feedback_log_body column to logs and backfill it from message for existing rows
  • capture span names plus formatted span and event fields at write time, since SQLite does not retain enough structure to reconstruct the old formatter later
  • keep SQLite feedback queries scoped to the requested thread plus same-process threadless rows
  • restore a SQL-side cumulative estimated_bytes cap for feedback export queries so over-retained partitions do not load every matching row before truncation
  • add focused formatting coverage for exported feedback lines and parity coverage against tracing_subscriber

Testing

  • cargo test -p codex-state
  • just fix -p codex-state
  • just fmt

codex author: codex resume 019ca1b0-0ecc-78b1-85eb-6befdd7e4f1f

@charley-oai
Copy link
Collaborator Author

@codex review this

Copy link
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 74b701c7de

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@charley-oai
Copy link
Collaborator Author

@codex review this

Copy link
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 79f633663c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@charley-oai
Copy link
Collaborator Author

@codex review this

Copy link
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 333d10ff31

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@charley-oai
Copy link
Collaborator Author

@codex review this

Copy link
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 459196a19e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@charley-oai
Copy link
Collaborator Author

@codex review this

@chatgpt-codex-connector
Copy link
Contributor

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@charley-oai charley-oai force-pushed the cc/feedback-log-body-rename branch 2 times, most recently from e746aae to 0891eb9 Compare March 6, 2026 00:55
@charley-oai
Copy link
Collaborator Author

@codex review this

@chatgpt-codex-connector
Copy link
Contributor

Codex Review: Didn't find any major issues. What shall we delve into next?

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@charley-oai charley-oai force-pushed the cc/feedback-log-body-rename branch from 0891eb9 to f91ab03 Compare March 6, 2026 19:54
charley-oai and others added 7 commits March 17, 2026 03:16
Store the rendered /feedback log body alongside each SQLite log row so exports keep span prefixes, structured fields, and timestamped legacy formatting. Rename the stored field to feedback_log_body to make it clear that timestamp and level are added later.

Co-authored-by: Codex <noreply@openai.com>
Include the larger of message and feedback_log_body when estimating per-partition log usage so span-prefixed or structured feedback bodies cannot bypass pruning, while preserving existing retention for ordinary message-only logs.

Co-authored-by: Codex <noreply@openai.com>
Drop the fixed SQL byte-overhead estimate and truncate on the exact formatted feedback lines after fetching the already-retained SQLite rows. Also remove single-use helper noise and add a focused formatting test to better match the existing feedback formatter contract.

Co-authored-by: Codex <noreply@openai.com>
Document that LOG_PARTITION_SIZE_LIMIT_BYTES and estimated_bytes bound retained log content for the message and feedback export read paths, rather than the exact sum of all persisted SQLite column bytes.

Co-authored-by: Codex <noreply@openai.com>
Explain that the pruning budget tracks reader-visible log content across the message and feedback export paths, and that we charge the larger payload instead of double-counting the same event.

Co-authored-by: Codex <noreply@openai.com>
Avoid appending an extra newline when feedback_log_body already ends with one so /feedback keeps the old SQL formatter behavior and byte accounting. Add a regression test for newline-terminated log bodies.

Co-authored-by: Codex <noreply@openai.com>
Move the feedback_log_body schema change into logs_migrations after the logs table moved to its own database, and update affected codex-state tests.

Co-authored-by: Codex <noreply@openai.com>
@charley-oai charley-oai force-pushed the cc/feedback-log-body-rename branch from f91ab03 to 81b8f9e Compare March 17, 2026 10:17
@charley-oai
Copy link
Collaborator Author

@codex review

@chatgpt-codex-connector
Copy link
Contributor

Codex Review: Didn't find any major issues. Can't wait for the next one!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

charley-oai and others added 3 commits March 17, 2026 04:09
Persist only feedback_log_body in the dedicated logs database and
route normal log reads/search through the rendered body.

Co-authored-by: Codex <noreply@openai.com>
Collapse the single-use span prefix helper into the feedback log body
formatter in log_db.

Co-authored-by: Codex <noreply@openai.com>
Update the sqlite state test to match rendered log bodies, which may include
span prefixes before the ToolCall text.

Co-authored-by: Codex <noreply@openai.com>
Remove the standalone process_uuid index from the feedback log migration and assert the final index set in the migration test.

Co-authored-by: Codex <noreply@openai.com>
@charley-oai charley-oai enabled auto-merge (squash) March 18, 2026 22:22
@charley-oai charley-oai merged commit ebbbc52 into main Mar 18, 2026
33 checks passed
@charley-oai charley-oai deleted the cc/feedback-log-body-rename branch March 18, 2026 22:44
@github-actions github-actions bot locked and limited conversation to collaborators Mar 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants