feat(ai): improve AI SDK v6 compatibility with DurableAgent by KaiKloepfer · Pull Request #928 · vercel/workflow

KaiKloepfer · 2026-02-04T06:29:18Z

Summary

Add AI SDK v6 (LanguageModelV3) compatibility to DurableAgent while maintaining full backward compatibility with AI SDK v5.

V3 stream normalization: Finish reasons ({unified, raw} objects), usage (nested {total, noCache, cacheRead, ...}), and new stream parts (preliminary tool results, tool-approval-request) are normalized to a shape compatible with both SDK versions.
tool-approval-request auto-deny: V3 tool approval requests are auto-denied with an execution-denied result so the agent loop continues instead of hanging.
Typed tool results: executeTool now passes through typed ToolResultOutput values (content, error-text, error-json, execution-denied) without re-wrapping, enabling multimodal tool results (images, files) to reach the model.
Workflow-safe download: Default download function now runs inside a step boundary where globalThis.fetch is available, fixing file/image URL fetching in workflow VM context and persisting results to the step log for replay.
instructions alias: Added instructions as an alias for system on both the constructor and stream() options, matching the AI SDK v6 ToolLoopAgent API.
totalUsage in onFinish: Aggregated token usage across all steps is now available in the onFinish callback.
Type safety improvements: Replaced StepResult<any> with StepResult<ToolSet>, correctly categorize static vs dynamic tool calls, extract filterToolSet to a shared module, centralize normalization logic in normalize.ts.

Closes #848

Partially addresses #168 (instructions alias)

Changes

File	What changed
`durable-agent.ts`	Workflow-safe download function, `instructions` alias, `totalUsage` in `onFinish`, typed tool result pass-through, type-safe casts
`normalize.ts` (new)	`normalizeFinishReason`, `normalizeUsage`, `addUsage`, `NormalizedUsage` type
`normalize.test.ts` (new)	Tests for all normalization functions (V2/V3 formats, edge cases)
`do-stream-step.ts`	`StreamableModel` interface, V3 preliminary tool results, `tool-input-start` extensions, `tool-approval-request` auto-deny, request metadata capture
`stream-text-iterator.ts`	`getToolOutputForUI` for V3 output types, use centralized normalization
`types.ts`	Remove incompatible `doStream` from V3 branch, add V3 extension interfaces
`filter-tools.ts` (new)	Extract shared `filterToolSet`
`durable-agent.test.ts`	Tests for `instructions`, typed tool results, `totalUsage`; use string model IDs
`stream-text-iterator.test.ts`	Tests for `getToolOutputForUI`
`do-stream-step.test.ts` (deleted)	Tests moved to `normalize.test.ts`
`durable-agent.mdx`	Instructions alias, tool result types, migration guide

@pranaygp, it looks like you might be the right maintainer for this?

Add LanguageModelV3 support while maintaining full v5 backward compatibility. Model objects (V2/V3) are auto-converted to string IDs for step boundary serialization. V3 finish reasons, usage, and stream parts are normalized to a shape both SDK versions can consume. Tool results with typed ToolResultOutput (content, error-text, error-json, execution-denied) now pass through without re-wrapping, enabling multimodal tool results (images, files) to reach the model. Also adds `instructions` alias for `system`, `totalUsage` in the `onFinish` callback, and improves type safety throughout (StepResult<any> → StepResult<ToolSet>, centralized normalization, shared filterToolSet). Closes vercel#848 Ref vercel#168

changeset-bot · 2026-02-04T06:29:23Z

🦋 Changeset detected

Latest commit: d1b0d9b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@workflow/ai	Patch
@workflow/docs-typecheck	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vercel · 2026-02-04T06:29:30Z

@KaiKloepfer is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

I, Kai Kloepfer <kai@biofire.io>, hereby add my Signed-off-by to this commit: add2232 Signed-off-by: Kai Kloepfer <kai@biofire.io>

pranaygp · 2026-02-04T20:42:58Z

+
+**String IDs** are the recommended approach. The string is resolved via the AI SDK gateway inside the workflow step, so it crosses the step boundary cleanly.
+
+**Model objects** from both AI SDK v5 (`LanguageModelV2`) and AI SDK v6 (`LanguageModelV3`) are accepted. They are automatically converted to string IDs using the object's `provider` and `modelId` fields. Middleware and wrappers applied to the model object are **not preserved** across the step boundary — use `providerOptions` to configure provider-specific settings instead.


How would this approach work? you can't serialize the doStream etc. functions inside LanguageModelV3 can't be serialized which is why we do the factory function version that allows us to hack around it

Addressed — reverted the model object auto-conversion in 2519749.

pranaygp · 2026-02-04T20:46:36Z

I don't think this approach works for LanguageModelV3. you can't just serialize it into a string like that since the object being serialized is indeed a function and needs to be rehydrated (otherwise you're just converting any other provider into a gateway string)

will need AI sdk to include "use step"/"use workflow" and/or implement custom serde probably

still discussing this with AI sdk so they can implement a proper solution

KaiKloepfer · 2026-02-05T04:25:42Z

Hey @pranaygp, thanks for the review!

I think this is worth discussing but wanted to point out that the factory function path on main (() => Promise<CompatibleLanguageModel>) doesn't actually work in workflow mode either, since doStreamStep is a 'use step' function and the function arg can't be serialized across the step boundary. So in practice, strings through gateway is already the only working path today.

Main already claims V3 compatibility in types.ts and DurableAgentOptions, this PR doesn't change that claim, it just makes the conversion explicit. If someone passes a model object, we convert it to a string with a warning instead of letting it silently fail at runtime. The V3 duck type with doStream on main is effectively unreachable.

Totally agree that a proper serde solution from the AI SDK would be the right long-term fix though. Happy to hold off on this if you'd rather wait for that.

I will note the model string conversion is a pretty small part of this PR, most of it is other fixes like V3 stream part normalization, usage aggregation, and finish reason handling that are currently missing.

pranaygp

Good PR — this correctly addresses the real bugs with V6 compatibility that were identified in #929. The centralized normalize.ts approach is clean, the typed tool result passthrough fixes a real gap, and the instructions alias + totalUsage are nice additions.

pranaygp · 2026-02-09T21:45:08Z

+  }
+
+  const obj = raw as Record<string, unknown>;
+


The V2 branch detection (typeof obj.inputTokens === 'number') is solid, but note that V2 also has totalTokens as a first-class field from the provider. The fallback inputTokens + outputTokens is fine as a safety net, but worth calling out that it may differ slightly from the provider-reported total (e.g., some providers include system tokens in totalTokens but not in inputTokens).

Good call. The code prefers the provider-reported totalTokens when available and only falls back to inputTokens + outputTokens, so it should pick up system tokens etc. when the provider includes them.

pranaygp · 2026-02-09T21:45:08Z

+  if (typeof result !== 'object' || result === null) return false;
+  const type = (result as Record<string, unknown>).type;
+  return typeof type === 'string' && TOOL_RESULT_OUTPUT_TYPES.has(type);
+}


This is a runtime duck-type check, which means a tool returning a plain object that happens to have { type: 'text', value: '...' } as its natural shape would be misinterpreted as a typed ToolResultOutput instead of getting wrapped as json. This is unlikely in practice, but worth documenting in the jsdoc that tools should not return objects with a type field matching these values unless they intend it to be a ToolResultOutput.

Fair point, added a JSDoc noting the caveat. Tools that happen to return { type: 'text', value: '...' } would get misinterpreted, but it's unlikely in practice.

pranaygp · 2026-02-09T21:45:08Z

+      `a string for step boundary serialization. Middleware and wrappers are not preserved. ` +
+      `Use providerOptions to configure provider-specific settings.`
+  );
+  return `${model.provider}/${model.modelId}`;


The resolveModelId function always logs a console.warn when a model object is passed. Since passing model objects is documented as "Accepted" (not deprecated), this warning may be noisy for users who intentionally use this pattern. Consider making the warning only fire once, or downgrading to a console.info/console.debug, or only warning if middleware is detected (though that's harder to check).

actually here's my comment: #928 (comment)

Moot now — resolveModelId was removed in the revert commit.

pranaygp · 2026-02-09T21:45:08Z

+// These interfaces centralize duck-typing assumptions for AI SDK v6
+// properties that don't exist in the installed @ai-sdk/provider V2 type
+// definitions. When @ai-sdk/provider ships V3 types, replace these with
+// the canonical imports and remove the casts in do-stream-step.ts.


Nice improvement — removing doStream from the V3 branch and only keeping the identity properties is the right call. This makes the duck type honest about what's actually used (just provider and modelId for string extraction), rather than pretending V3 accepts V2 call options.

pranaygp · 2026-02-09T21:45:08Z

+                );
+              }
            }
          }


The tool-approval-request warning is a good addition. However, this will currently just warn and then the tool call will sit there with no response — the model may hang or timeout. Consider whether it's feasible to auto-deny the approval and return an error result so the agent loop can continue, rather than silently stalling.

Good catch. Changed this to auto-deny with an execution-denied tool result so the agent loop can continue instead of hanging. The handling moved to the first TransformStream so the denied result flows through properly. Still logs a warning.

pranaygp · 2026-02-09T21:45:09Z

+    const obj = raw as Record<string, unknown>;
+    const rawValue = typeof obj.raw === 'string' ? obj.raw : undefined;
+    // V3: { unified: 'stop', raw: 'stop' }
+    if (typeof obj.unified === 'string') {


Edge case: if raw is an empty string "", this currently maps to { finishReason: 'unknown', rawFinishReason: undefined } (because of the raw === '' ? 'unknown' : raw check). But then rawFinishReason is set to raw || undefined which also converts empty string to undefined. This is probably fine since no real provider sends an empty string finish reason, but the divergence between the finishReason path (explicit empty check) and rawFinishReason path (falsy check) is a bit inconsistent.

Yeah the two paths are slightly inconsistent (=== '' vs falsy check) but converge to the same result for any real input. Could tighten it but didn't want to over-engineer for a case no provider hits.

pranaygp · 2026-02-09T21:45:09Z

+                  result: chunk.result,
+                  isError: chunk.isError,
+                });
+              }


The preliminary result handling logic stores the preliminary result only if no result exists yet, and overwrites with the final one. This means if multiple preliminary results arrive for the same tool call, only the first one is kept. Is that intentional? The AI SDK's own handling of preliminary results streams them all through. Might be fine for DurableAgent's use case (we only need the final result), just flagging.

Intentional — I only need the final result, and the preliminary is just a placeholder until it arrives. If multiple preliminaries come in, the first one is fine since it gets overwritten by the final anyway.

pranaygp · 2026-02-09T21:45:09Z

+      if (o.type === 'execution-denied') {
+        return o.reason ?? 'Tool execution was denied';
+      }
+      return output;


getToolOutputForUI is clean. One thought: the execution-denied branch falls through to a runtime check rather than being in the type-safe switch — when @ai-sdk/provider adds V3 output types, this should get a proper case arm. The comment already flags this, so just a +1 on the TODO.

pranaygp · 2026-02-09T21:55:54Z

I think this is worth discussing but wanted to point out that the factory function path on main (() => Promise<CompatibleLanguageModel>) doesn't actually work in workflow mode either, since doStreamStep is a 'use step' function and the function arg can't be serialized across the step boundary. So in practice, strings through gateway is already the only working path today.

I don't think this is true. @TooTallNate can you confirm? The function arg does get serialized across step boundaries, but it's a bit of a hack because we're actually relying on bundling behaviour. i.e. the functions in CompatibleLanguageModel are actually registered alongside the step, so when we pass the step function callback as an argument into doStreamStep, it works because step functions have been specifically special cased to work across serde. this means doStreamStep does call the actual correct model provider function that's accessible in the step bundle

If you follow the guide here and use "Custom provider" and attempt to use one of the custom providers that we export through the @workflow/ai package (for example), you shuold be able to create a DurableAgent without using gateway

we certainly don't want to break that functionality

I think the rest of the PR is something we should get in but I'm very hesitant on magically converting custom providers into a gateway string

✅ Review feedback validated by Claude

Confirmed by reviewing the serialization system and SWC plugin output:

StepFunction serde is real: packages/core/src/serialization.ts:567-582 has explicit StepFunction reducers that detect functions with a stepId property and serialize only the stepId + closure variables (not the function body). The reviver at line 1137 looks up the function via getStepFunction(stepId) in the step registry. There's a dedicated test at serialization.test.ts:1752 — "should dehydrate step function passed as argument to a step".
All @workflow/ai providers use this pattern: Every provider (openai, anthropic, google, xai, gateway) returns an async function with 'use step' — the SWC plugin assigns it a stepId and registers it via registerStepFunction(). When passed as the modelInit arg to doStreamStep (also a 'use step' function), the step function reference survives the boundary because only the stepId is persisted to the event log.
The PR's resolveModelId breaks this: It deprecates factory functions with a console.warn, but factory functions are the primary mechanism for custom providers. It also converts model objects to "provider/modelId" gateway strings, which silently drops custom API keys, base URLs, middleware, and any provider config not expressible through providerOptions.
The rest of the PR is solid: The centralized normalize.ts, tool result passthrough (isToolResultOutput), instructions alias, and totalUsage all address real bugs and should land — it's specifically the model resolution changes that need rework.

The AI SDK's default download function uses globalThis.fetch, which is unavailable in the workflow VM context. Add a step-based download function so file/image URLs in prompts are fetched inside the step boundary where globalThis.fetch works, and results are persisted to the step log for replay on workflow resumption. Signed-off-by: Kai Kloepfer <kai@biofire.io>

Factory functions (() => Promise<LanguageModel>) work across step boundaries due to bundling behavior, so they should not be deprecated. Remove resolveModelId which magically converted model objects to gateway strings, and restore the original model type signature. Signed-off-by: Kai Kloepfer <kai@biofire.io>

Address maintainer review feedback: - Auto-deny V3 tool-approval-request with execution-denied result instead of just warning, preventing the agent loop from hanging - Add JSDoc to isToolResultOutput documenting the duck-type collision risk Signed-off-by: Kai Kloepfer <kai@biofire.io>

KaiKloepfer · 2026-02-13T03:37:00Z

Thanks @pranaygp, I haven't reviewed the bundling side in depth at all, so can't comment there.

The factory function issue is why I started to dig into this in the first place, I've been trying (basically unsuccessfully due to a series of limitations) as a vercel enterprise customer to get a basic production feature set (observability, prompt management, durable retries, tool approvals, mcp calls, etc.) impl done with workflow, ai sdk and deployed to vercel. This was the first major issue I encountered.

That being said, I followed your link and it DID work. Commit pushed to revert, address a few of your other topics, and to fix a further issue, see 61c5347.

Here is the pattern I was running into as a user that DOES NOT work:

This works:

this.agent = new DurableAgent({
  model: config.gatewayId,
...
})

This causes the below errors:

const gatewayId = config.gatewayId
this.agent = new DurableAgent({
  model: async () => {
    console.debug('[workflow-agent] model factory called (host context)')
    return gateway(gatewayId)
  },
...
})

Errors

Errors in local world:

[Workflow] Serialization failed { context: 'step arguments', problematicValue: [AsyncFunction: model] }
 POST /.well-known/workflow/v1/flow 500 in 150ms (compile: 1825µs, render: 148ms)
Error [WorkflowRuntimeError]: Failed to serialize step arguments at path ".args[1]". Ensure you're passing serializable types (plain objects, arrays, primitives, Date, RegExp, Map, Set).

Learn more: https://useworkflow.dev/err/serialization-failed
    at ignore-listed frames {
  [cause]: Error [DevalueError]: Cannot stringify a function
      at ignore-listed frames {
    path: '.args[1]',
    value: [AsyncFunction: model],
    root: { args: [Array], closureVars: undefined, thisVal: undefined }
  }
}
⨯ unhandledRejection: Error [WorkflowRuntimeError]: Failed to serialize step arguments at path ".args[1]". Ensure you're passing serializable types (plain objects, arrays, primitives, Date, RegExp, Map, Set).

Learn more: https://useworkflow.dev/err/serialization-failed
    at ignore-listed frames {
  [cause]: Error [DevalueError]: Cannot stringify a function
      at ignore-listed frames {
    path: '.args[1]',
    value: [AsyncFunction: model],
    root: { args: [Array], closureVars: undefined, thisVal: undefined }
  }
}
⨯ unhandledRejection:  Error [WorkflowRuntimeError]: Failed to serialize step arguments at path ".args[1]". Ensure you're passing serializable types (plain objects, arrays, primitives, Date, RegExp, Map, Set).

Learn more: https://useworkflow.dev/err/serialization-failed
    at ignore-listed frames {
  [cause]: Error [DevalueError]: Cannot stringify a function
      at ignore-listed frames {
    path: '.args[1]',
    value: [AsyncFunction: model],
    root: { args: [Array], closureVars: undefined, thisVal: undefined }
  }
}
[local world] Failed to queue message {
  queueName: '__wkf_workflow_workflow//./src/lib/ai/workflows/chat//chat',
  text: `"WorkflowRuntimeError: Failed to serialize step arguments at path \\".args[1]\\". Ensure you're passing serializable types (plain objects, arrays, primitives, Date, RegExp, Map, Set).\\n\\nLearn more: https://useworkflow.dev/err/serialization-failed"`,
  status: 500,
  headers: {
    connection: 'keep-alive',
    'content-type': 'application/json',
    date: 'Fri, 13 Feb 2026 02:52:57 GMT',
    'keep-alive': 'timeout=5',
    'transfer-encoding': 'chunked',
    vary: 'rsc, next-router-state-tree, next-router-prefetch, next-router-segment-prefetch'
  },
  body: '{"runId":"wrun_01KHAEMVW7QFN9094SDRS55RNK","traceCarrier":{"traceparent":"00-de212ac519cfc889e5b71844427ad77c-839138c0be8df326-01","baggage":"workflow.run_id=wrun_01KHAEMVW7QFN9094SDRS55RNK,workflow.name=workflow%2F%2F.%2Fsrc%2Flib%2Fai%2Fworkflows%2Fchat%2F%2Fchat"},"requestedAt":"2026-02-13T02:52:57.249Z"}'
}

What does work is something like:

import { gateway } from '@workflow/ai/gateway'

/**
 * Get a serializable model factory for DurableAgent.
 *
 * Uses `@workflow/ai/gateway` which returns `() => Promise<LanguageModelV2>` where
 * the inner function has `'use step'` — making it serializable across the workflow
 * step boundary. 
 */
export function getDurableModel(name: ModelName): () => Promise<import('@ai-sdk/provider').LanguageModelV2> {
  const config = MODEL_CONFIGS[name]
  return gateway(config.gatewayId)
}

this.agent = new DurableAgent({
      model: getDurableModel(options.model),
      ...

Add early return after enqueuing the auto-denied tool result so the original tool-approval-request chunk is not also pushed to the stream. Signed-off-by: Kai Kloepfer <kai@biofire.io>

VaguelySerious · 2026-02-26T18:45:09Z

@pranaygp @KaiKloepfer Are there still changes being made here or is it ready for re-review?

DCO Remediation Commit for Kai Kloepfer <kai@biofire.io>

3a9265d

I, Kai Kloepfer <kai@biofire.io>, hereby add my Signed-off-by to this commit: add2232 Signed-off-by: Kai Kloepfer <kai@biofire.io>

KaiKloepfer mentioned this pull request Feb 4, 2026

🆕 WorkflowAgent (@ai-sdk/workflow) vercel/ai#12165

Merged

5 tasks

pranaygp reviewed Feb 4, 2026

View reviewed changes

pranaygp reviewed Feb 9, 2026

View reviewed changes

pranaygp mentioned this pull request Feb 9, 2026

Add provider wrappers for all official AI SDK providers #986

Closed

3 tasks

KaiKloepfer added 3 commits February 12, 2026 20:15

vercel Bot reviewed Feb 13, 2026

View reviewed changes

Comment thread packages/ai/src/agent/do-stream-step.ts

fix(ai): prevent tool-approval-request chunk from leaking into stream

d1b0d9b

Add early return after enqueuing the auto-denied tool result so the original tool-approval-request chunk is not also pushed to the stream. Signed-off-by: Kai Kloepfer <kai@biofire.io>


		String IDs are the recommended approach. The string is resolved via the AI SDK gateway inside the workflow step, so it crosses the step boundary cleanly.

		Model objects from both AI SDK v5 (`LanguageModelV2`) and AI SDK v6 (`LanguageModelV3`) are accepted. They are automatically converted to string IDs using the object's `provider` and `modelId` fields. Middleware and wrappers applied to the model object are not preserved across the step boundary — use `providerOptions` to configure provider-specific settings instead.

Conversation

KaiKloepfer commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Uh oh!

changeset-bot Bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

vercel Bot commented Feb 4, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pranaygp commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KaiKloepfer commented Feb 5, 2026

Uh oh!

pranaygp left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pranaygp commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KaiKloepfer commented Feb 13, 2026

Uh oh!

Uh oh!

VaguelySerious commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

KaiKloepfer commented Feb 4, 2026 •

edited

Loading

changeset-bot Bot commented Feb 4, 2026 •

edited

Loading

pranaygp commented Feb 4, 2026 •

edited

Loading

pranaygp left a comment •

edited

Loading

pranaygp commented Feb 9, 2026 •

edited

Loading