Skip to content

Comments

feat(provider/azure) : enable codeInterpreter and fileSearch for AzureOpenAI#8238

Merged
gr2m merged 41 commits intovercel:mainfrom
tsuzaki430:tsuz/azure-responsesapi-code-interpreter
Sep 26, 2025
Merged

feat(provider/azure) : enable codeInterpreter and fileSearch for AzureOpenAI#8238
gr2m merged 41 commits intovercel:mainfrom
tsuzaki430:tsuz/azure-responsesapi-code-interpreter

Conversation

@tsuzaki430
Copy link
Collaborator

@tsuzaki430 tsuzaki430 commented Aug 23, 2025

Background

The built-in tool 'code_interpreter' and 'file_search' are enabled on Azure OpenAI Responses API.

tool/provider OpenAI Azure OpenAI
web_search ✅ Enabled ❌ Not currently supported
code_interpreter ✅ Enabled ✅ Enabled
file_search ✅ Enabled ✅ Enabled

note : currently, web_search is not supported in Azure OpenAI provider.
https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/responses?tabs=python-secure#model-support

Summary

Azure provider's code_interpreter and file_search is enabled same as openai provider.
openai's tools code_interpreter and file_search are added to internal export (packages/openai/src/internal/index.ts).
And azure's tools get the implementations from import openai provider's tools.

azure.tools.fileSearch build-in tool will be available.

const result = await generateText({
  model: azure.responses('gpt-4.1-mini'),
  prompt: 'What does the document say about user authentication?',
  tools: {
    file_search: azure.tools.fileSearch({
      // optional configuration:
      vectorStoreIds: ['vs_123', 'vs_456'],
      maxNumResults: 10,
      ranking: {
        ranker: 'auto',
      },
    }),
  },
  // Force file search tool:
  toolChoice: { type: 'tool', toolName: 'file_search' },
});

azure.tools.codeIinterpreter build-in tool also will be available.

const result = await generateText({
  model: azure.responses('gpt-4.1-mini'),
  prompt: 'Write and run Python code to calculate the factorial of 10',
  tools: {
    code_interpreter: azure.tools.codeInterpreter({
      // optional configuration:
      container: {
        fileIds: ['assistant-123', 'assistant-456'], // optional file IDs to make available
      },
    }),
  },
});

Manual Verification on Azure

✅code_interpreter works.
✅code_interpreter with fileIds "assistant-" prefix files.
✅file_search works is "vs_" vector store.
✅both openai and azure providers , I tried same test and get almost the same result.

Others

I met the file_search zod parse error on openai and azure provider both environments.
#8030 (comment)
When it will be fixed on openai , I think azure's tool will be fixed , too.

Tasks

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • Formatting issues have been fixed (run pnpm prettier-fix in the project root)
  • I have reviewed this pull request (self-review)

tsuzaki430 and others added 2 commits August 27, 2025 22:48
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
@tsuzaki430 tsuzaki430 marked this pull request as ready for review August 27, 2025 14:12
Copy link
Collaborator Author

@tsuzaki430 tsuzaki430 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My own review finished. Thank you.

@tsuzaki430 tsuzaki430 changed the title feat(provider/azure) : enable code_interpreter and file_search in Responses API feat(provider/azure) : enable codeInterpreter and fileSearch for Azure OpenAI Aug 30, 2025
@tsuzaki430 tsuzaki430 changed the title feat(provider/azure) : enable codeInterpreter and fileSearch for Azure OpenAI feat(provider/azure) : enable codeInterpreter and fileSearch for AzureOpenAI Aug 30, 2025
@gr2m
Copy link
Collaborator

gr2m commented Sep 7, 2025

Hi @tommasoghisini and @jephal I've seen you reacting to the PR. Can you confirm that the codeInterpreter and fileSearch is working with @tsuzaki430's changes? I have trouble with my own local test environment and Azure right now. I'd like to get this shipped though 😁

Copy link
Collaborator

@gr2m gr2m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tsuzaki430 thank you for another great pull request! Changes look great, I just need to confirm it's working as it should, or have someone else confirm.

Could you please add two examples for the Code Interpreter and File Search features? You can use https://github.com/vercel/ai/blob/861fb35ccd8557959f8804230aacca07ba9f8bd3/examples/ai-core/src/generate-text/azure.ts as a template

@jephal
Copy link
Collaborator

jephal commented Sep 8, 2025

Hi @gr2m @tsuzaki430, tested locally with a generateText call and got a few errors:

  1. when calling with 2 vectorstores it only gives a response based on the last one passed in the array, and doesn't read the first value. (2 vectorstores are max but this seems to be by design). Also i get this warning AI SDK Warning: tool result parts in assistant messages are not supported for OpenAI responses

  2. when running code-interpreter i get a type error. in the output response i get a response of type 'code_interpreter_call', could this be as it doesnt match any of the message parts/types 'text', 'tool_call' etc?
    "output": [ { "id": "ci_12345", "type": "code_interpreter_call", "status": "completed", "code": "# Generate and execute a Python script that counts Fibonacci numbers up to 1000\

@tsuzaki430
Copy link
Collaborator Author

tsuzaki430 commented Sep 26, 2025

Hello , dear @gr2m 😊
Thank you for your daily maintenance of the AI ​​SDK.
I recently encountered a conflict, but fixed it and it now works as expected.
We're really looking forward to seeing file-search and code-interpreter working with the AI ​​SDK on Azure!🚀
Could you please review this PR when you have time?✨

Copy link
Collaborator

@gr2m gr2m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go, thank you all!

@gr2m gr2m added the backport label Sep 26, 2025
@gr2m gr2m merged commit 77f2b20 into vercel:main Sep 26, 2025
13 of 14 checks passed
vercel-ai-sdk bot pushed a commit that referenced this pull request Sep 26, 2025
…eOpenAI (#8238)

## Background

The built-in tool 'code_interpreter' and 'file_search' are enabled on
Azure OpenAI Responses API.


tool/provider | OpenAI | Azure OpenAI
-- | -- | --
web_search | ✅ Enabled | ❌ Not currently supported
code_interpreter | ✅ Enabled | ✅ Enabled
file_search | ✅ Enabled | ✅ Enabled

note : currently, web_search is not supported in Azure OpenAI provider.

https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/responses?tabs=python-secure#model-support

## Summary
Azure provider's code_interpreter and file_search is enabled same as
openai provider.
openai's tools code_interpreter and file_search are added to internal
export (packages/openai/src/internal/index.ts).
And azure's tools get the implementations from import openai provider's
tools.

azure.tools.fileSearch build-in tool will be available.

``` typescript
const result = await generateText({
  model: azure.responses('gpt-4.1-mini'),
  prompt: 'What does the document say about user authentication?',
  tools: {
    file_search: azure.tools.fileSearch({
      // optional configuration:
      vectorStoreIds: ['vs_123', 'vs_456'],
      maxNumResults: 10,
      ranking: {
        ranker: 'auto',
      },
    }),
  },
  // Force file search tool:
  toolChoice: { type: 'tool', toolName: 'file_search' },
});
```

azure.tools.codeIinterpreter build-in tool also will be available.
``` typescript
const result = await generateText({
  model: azure.responses('gpt-4.1-mini'),
  prompt: 'Write and run Python code to calculate the factorial of 10',
  tools: {
    code_interpreter: azure.tools.codeInterpreter({
      // optional configuration:
      container: {
        fileIds: ['assistant-123', 'assistant-456'], // optional file IDs to make available
      },
    }),
  },
});
```

## Manual Verification on Azure
✅code_interpreter works.
✅code_interpreter with fileIds "assistant-" prefix files.
✅file_search works is "vs_" vector store.
✅both openai and azure providers , I tried same test and get almost the
same result.

## Others
I met the file_search zod parse error on openai and azure provider both
environments.
#8030 (comment)
When it will be fixed on openai , I think azure's tool will be fixed ,
too.

## Tasks
- [x] Tests have been added / updated (for bug fixes / features)
- [x] Documentation has been added / updated (for bug fixes / features)
- [x] A _patch_ changeset for relevant packages has been added (for bug
fixes / features - run `pnpm changeset` in the project root)
- [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the
project root)
- [x] I have reviewed this pull request (self-review)

---------

Co-authored-by: tsuzaki430 <tsuzaki430@users.noreply.github.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
Co-authored-by: Gregor Martynus <39992+gr2m@users.noreply.github.com>
@vercel-ai-sdk vercel-ai-sdk bot removed the backport label Sep 26, 2025
@vercel-ai-sdk
Copy link
Contributor

vercel-ai-sdk bot commented Sep 26, 2025

✅ Backport PR created: #8962

@gr2m
Copy link
Collaborator

gr2m commented Sep 26, 2025

@tommasoghisini @tsuzaki430 @jephal I'm looking into creating a @vercel/ai-sdk-azure team. It would be our first community maintainer team. Would any of you be interested to help out? No expectations, it's just an experiment at this point. But the idea is that we would prioritize pull requests for the azure provider that is either created or approved by a member of the @vercel/ai-sdk-azure team

gr2m added a commit that referenced this pull request Sep 26, 2025
…h for AzureOpenAI (#8962)

This is an automated backport of #8238 to the release-v5.0 branch.

Co-authored-by: tsuzaki430 <58161200+tsuzaki430@users.noreply.github.com>
Co-authored-by: tsuzaki430 <tsuzaki430@users.noreply.github.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
Co-authored-by: Gregor Martynus <39992+gr2m@users.noreply.github.com>
@rahulbhadja
Copy link
Collaborator

@gr2m , we've been using Azure OpenAI for the past 2 years and would be happy to contribute to the ai-sdk project. We're also currently using @ai-sdk/azure in our application, so I'd be glad to contribute and help improve the Azure integration as well.

@tsuzaki430 tsuzaki430 deleted the tsuz/azure-responsesapi-code-interpreter branch September 27, 2025 02:40
@tsuzaki430
Copy link
Collaborator Author

tsuzaki430 commented Sep 27, 2025

@gr2m

Hello,
Thank you for review and new release.

Would any of you be interested to help out?

Yes, maybe I can do something help.
Thank you for your message.
I like think of AI SDK.

@tommasoghisini
Copy link
Collaborator

tommasoghisini commented Sep 28, 2025

@gr2m I really like the idea. Support for the Azure provider is key for us, so I'm more than happy to help out.

Thanks for putting this together!

@jephal
Copy link
Collaborator

jephal commented Sep 29, 2025

@gr2m Sounds great, Azure support for AI-sdk is very important to our team :)

@gr2m
Copy link
Collaborator

gr2m commented Sep 30, 2025

@rahulbhadja @tsuzaki430 you should have gotten invitations.

@tommasoghisini @jephal coming soon, thank you for offering to help!

@rahulbhadja
Copy link
Collaborator

Hi @gr2m , I got the invite but it's asking for Okta login that I don't have, and not able to reset pass. Can you help me with it? Thanks!

@tsuzaki430
Copy link
Collaborator Author

Hello,
Thank you for invite.
Same as Rahul , I don't have Okta login.
So I couldn't input correct username and password.

@gr2m
Copy link
Collaborator

gr2m commented Oct 1, 2025

Sorry for that! It's the first time that @vercel is inviting external contributors. I'll find out if we can disable the Okta login for you

@tsuzaki430
Copy link
Collaborator Author

@gr2m
Hello , I received the invite and finished successfully.
Thank you and I want to contribute for @ai-sdk/azure , AI SDK.

@gr2m
Copy link
Collaborator

gr2m commented Oct 3, 2025

great, thank you for confirming. You should all have been invited by now, and shouldn't need an Okta login.

@jephal
Copy link
Collaborator

jephal commented Oct 3, 2025

Hi @gr2m, does that include @tommasoghisini and I? I cannot see an invite anywhere

@gr2m
Copy link
Collaborator

gr2m commented Oct 28, 2025

Hi all, apologies but while we were able to create the @vercel/ai-sdk-azure team, it turns out we cannot add external collaborators to it. I've been thinking what else I could do to keep everyone interested in the loop.

For now, I created an issue that we can utilize to communicate:
#9863

Hi @gr2m, does that include @tommasoghisini and I? I cannot see an invite anywhere

sorry not sure what happened. Can you see if https://github.com/vercel/ai/invitations works for you know?

@tommasoghisini
Copy link
Collaborator

Hi @gr2m, I believe the invite now worked :)

Will use #9863 to coordinate

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants