fix(ai-chat): serialize chat turns and saveMessages#1142
Merged
Conversation
🦋 Changeset detectedLatest commit: 78332f2 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
Introduce a monotonic _chatEpoch (incremented on CF_AGENT_CHAT_CLEAR) so queued continuations or save operations enqueued under an older epoch are skipped after a chat clear. Wire apply/approval promises as prerequisites for auto-continuations (catching rejections) so continuations only run if the tool result/approval was applied, and check epoch before running exclusive chat turns. Snap clientTools/body in saveMessages and avoid running them if the epoch changed. Minor cleanup: remove an unnecessary _getAbortSignal call in cancel flow. Add tests and test helpers (isChatTurnActiveForTest, waitForIdleForTest, persistToolCallMessage, getMessageCount) to cover tool-result continuations, chat-clear skipping, and saveMessages behavior; replace some fixed delays with waitForIdleForTest.
10f6a03 to
78332f2
Compare
threepointone
approved these changes
Mar 22, 2026
Contributor
threepointone
left a comment
There was a problem hiding this comment.
looks good. added some checks for an edge cases and added some tests. landing after ci passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1110
Summary
onChatMessage()+_reply()behind a private turn queue so websocket requests, tool auto-continuations, andsaveMessages()cannot stream concurrentlyisChatTurnActive(),waitForIdle(),abortActiveTurn(),hasPendingInteraction(), andwaitForPendingInteractionResolution()helpers for subclass code that needs to inspect active turns and defer injected work until pending tool interactions resolvesaveMessages()calls that were enqueued before a chat clear (epoch guard)saveMessages()context (_lastClientTools,_lastBody) at enqueue time so a later request cannot overwrite it before execution_queueAutoContinuationto use the caller-providedclientToolsdirectly instead of re-readingthis._lastClientToolsat execution time.then()microtask, closing a race wherewaitForIdle()could resolve before the continuation was queued_applyToolResult/_applyToolApprovalsaveMessages(), abort behavior, continuation body isolation, pending-interaction coordination, clear-during-active-turn, and clear-during-queued-saveMessagesDesign Notes
AIChatAgentalready assumes a single active resumable stream, so this PR keeps that model and serializes the higher-level chat turn (onChatMessage()+_reply()) instead of introducing concurrent stream support._streamCompletionPromise: it resolved too early for this use case because it did not guarantee final assistant-message persistence._chatEpochcounter is incremented on chat clear. Queued turns check the epoch before executing and skip if it changed, preventing stale continuations from writing into a cleared conversation. The counter is in-memory only — it doesn't need to survive hibernation because the queue itself (_chatTurnQueue) is in-memory and resets on wake.saveMessages()itself still only solves turn serialization; callers that need Claude Code-style queued/scheduled behavior can combinewaitForIdle()withwaitForPendingInteractionResolution()before injecting a synthetic message.Behavior Changes
saveMessages()now waits for any active turn, runs its own turn exclusively, and resolves only after the reply finishes.saveMessages()calls that were enqueued before the clear.isChatTurnActive(),waitForIdle(),abortActiveTurn()) support turn coordination.hasPendingInteraction(),waitForPendingInteractionResolution()) let subclasses detect unresolved client-tool input/approval state before starting a follow-up turn.Non-Goals / Limitations
abortActiveTurn()aborts the active request or stream, but it does not interrupt unrelated pre-stream setup such aswaitForMcpConnections().saveMessages()does not block on user interactions by default.Review Guide
packages/ai-chat/src/index.ts: queueing model, turn helpers, epoch guard, and pending-interaction helperspackages/ai-chat/src/tests/chat-turn-serialization.test.ts: websocket/saveMessages/abort ordering, clear-during-turn, tool-continuation coveragepackages/ai-chat/src/tests/custom-body-continuation.test.ts: continuation context isolationpackages/ai-chat/src/tests/pending-interaction.test.ts: pending interaction detection and waitingTesting
npm installnpm run buildnpm run test:workers --workspace @cloudflare/ai-chat -- src/tests/chat-turn-serialization.test.ts src/tests/custom-body-continuation.test.ts src/tests/pending-interaction.test.tsnpm run check