Skip to content

Update CachingChatClient with callbacks for if caching should happen #6030

@stephentoub

Description

@stephentoub

The caching chat client doesn't work great when dealing with an OpenAI Assistant-like entity, where the ConversationId is non-null and remains the same across all operations. In such cases, the provided chat messages and options are insufficient to uniquely identify state in the cache.

We should:

  1. Disable caching by default when ChatOptions.ConversationId is set
  2. Provide either virtual methods or delegate properties that allow determining on a per-call basis whether to check the cache or cache a result

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-aiMicrosoft.Extensions.AI libraries

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions