feat: recover from context overflow during agent steps#40
Open
MohamedAkbarally wants to merge 3 commits intofix/normalize-embedded-json-tool-args-flagfrom
Open
feat: recover from context overflow during agent steps#40MohamedAkbarally wants to merge 3 commits intofix/normalize-embedded-json-tool-args-flagfrom
MohamedAkbarally wants to merge 3 commits intofix/normalize-embedded-json-tool-args-flagfrom
Conversation
Retry the same turn after summarizing when the client reports a context overflow so agent runs can continue past provider context-window failures. Made-with: Cursor
MohamedAkbarally
changed the title
Retry message summarization on a shorter history by preserving the newest assistant-led turn, and raise a clear ContextOverflowError when nothing remains to peel. Made-with: Cursor
Collaborator
declanjackson
left a comment
declanjackson
left a comment
There was a problem hiding this comment.
LGTM - maybe we should quickly mention this logic in concepts.md?
Comment on lines
+1148
to
+1150
| task_context: list[ChatMessage] = list( | ||
| takewhile(lambda m: not isinstance(m, (AssistantMessage, SummaryMessage)), messages_to_summarize) | ||
| ) |
Collaborator
There was a problem hiding this comment.
This can be out of the loop right?
| # UserMessage (not AssistantMessage) to avoid consecutive assistant messages which some providers reject | ||
| acknowledgement_msg = UserMessage(content="Got it, thanks!") | ||
| messages_to_summarize, latest_suffix = split_messages | ||
| preserved_tail = [*latest_suffix, *preserved_tail] |
Collaborator
There was a problem hiding this comment.
I wonder if we log every time we peel for observability? Or maybe just build in num messages peeled into the summarization logging
Reset token usage on assistant messages copied into post-summary trajectories so history snapshots do not double count preserved turns. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ContextOverflowErrorin the agent loop and summarize once before retrying the same turnContextOverflowErrortranslations to trigger end-to-end overflow recovery