Skip to content

Comments

feat (ui/react): support resuming an ongoing stream#6000

Closed
jeremyphilemon wants to merge 16 commits intomainfrom
jrmy/experimental-resume
Closed

feat (ui/react): support resuming an ongoing stream#6000
jeremyphilemon wants to merge 16 commits intomainfrom
jrmy/experimental-resume

Conversation

@jeremyphilemon
Copy link
Contributor

@jeremyphilemon jeremyphilemon commented Apr 25, 2025

Background

This pull request adds the ability for clients to resume an ongoing chat generation stream after a network disconnect.

Summary

This pull request adds support for the useChat hook to resume an ongoing chat generation stream by exposing experimental_resume() that can be called by any client, typically during the initial mount of the hook.

The experimental_resume function makes a GET request to the api endpoint you've initialized the hook with (or /api/chat by default) and streams the contents of the stream if it is active or fails silently if it has ended.

In order for experimental_resume to work as intended, it requires the usage of the resumable-stream package for stream creation and a redis instance for the package to manage the pub/sub mechanism.

Verification

An example has been added at examples/next-openai/app/use-chat-resume to test the feature. Follow the following steps to test it end-to-end:

  1. Run the development server
  2. Navigate to http://localhost:3000/use-chat-resume
  3. Send a message that will have a longer generation duration, example "Write an essay about Michael Jordan"
  4. Once the generation starts, click the chat id above to open the conversation in a new tab
  5. Verify the stream gets resumed

Tasks

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added
  • Formatting issues have been fixed (run pnpm prettier-fix in the project root)

Future Work

Possibly unify resumeChatApi and callChatApi and avoid code duplication.

@jeremyphilemon jeremyphilemon force-pushed the jrmy/experimental-resume branch from 5e9426f to 89b2f75 Compare April 25, 2025 18:07

To resume a chat stream, you can use the `experimental_resume` function returned by the `useChat` hook. You will typically call this function during the initial mount of the hook.

```
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

language

Copy link
Contributor Author

@jeremyphilemon jeremyphilemon Apr 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, but can you verify since it was unclear what to improve

@lgrammel
Copy link
Collaborator

CC @iteratetograceness for visibility regarding the useChat chat store refactoring.

@jeremyphilemon
Copy link
Contributor Author

Created a new pull request since the tests here have bugged out #6052

@lgrammel lgrammel deleted the jrmy/experimental-resume branch July 2, 2025 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants