Skip to content

fix: support chat response data types enabling streaming of tool calls#32

Merged
adubovik merged 4 commits intodevelopmentfrom
fix/streaming-tool-calls
Jul 14, 2025
Merged

fix: support chat response data types enabling streaming of tool calls#32
adubovik merged 4 commits intodevelopmentfrom
fix/streaming-tool-calls

Conversation

@adubovik
Copy link
Copy Markdown
Collaborator

@adubovik adubovik commented Jul 14, 2025

Resolves #31

Breaking changes

Dependencies

  • bump lower bound on openai from 1.0.0 to 1.1.0, since 1.1.0 is the first version supporting tools.

Streaming chat completion response class

  • changed the type of delta field in aidial_client.types.chat.response.ChoiceDelta datatype to accomodate streaming tool call chunks:
class ChoiceDelta(ExtraAllowModel):
    index: int
-    delta: Union[ChatCompletionMessage, ChunkEmptyDelta]
+    delta: ChatCompletionMessageDelta
    finish_reason: Optional[str] = None
  • removed aidial_client.types.chat.response.ChunkEmptyDelta as redundant

@adubovik adubovik self-assigned this Jul 14, 2025
@adubovik adubovik force-pushed the fix/streaming-tool-calls branch from cbc24b6 to 5fffd5b Compare July 14, 2025 10:00
@adubovik adubovik requested a review from sdryapko July 14, 2025 10:31
@adubovik adubovik merged commit 2fc74c5 into development Jul 14, 2025
11 checks passed
@adubovik adubovik deleted the fix/streaming-tool-calls branch July 14, 2025 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failed to validate chunk with tool call in stream mode

2 participants