docs: add modalities configuration for custom providers#9898
docs: add modalities configuration for custom providers#9898slycrel wants to merge 2 commits intoanomalyco:devfrom
Conversation
- Document the 'modalities' property for enabling vision/multimodal support - Add example showing image input configuration - List all supported input/output modalities - Explain validation behavior without modalities config Fixes the discoverability issue where users cannot find how to enable vision support for custom OpenAI-compatible providers.
|
The following comment was made by an LLM, it may be inaccurate: Based on my search, I found one potentially related PR: PR #5531: Feature/OpenAI compatible reasoning This PR is mentioned in the current PR's description as relating to multimodal support for However, this is not a duplicate—it addresses a different feature (reasoning capabilities) rather than documentation of modalities configuration. The current PR #9898 is a documentation-only change that documents existing functionality. No duplicate PRs found that are currently addressing the same documentation scope of modalities configuration for custom providers. |
|
Related issue: #9897 |
|
Closes #9897 |
00637c0 to
71e0ba2
Compare
f1ae801 to
08fa7f7
Compare
docs: add modalities configuration for custom providers
Summary
Adds documentation for the
modalitiesproperty that enables vision and multimodal support in custom OpenAI-compatible providers.Problem
The
modalitiesconfiguration property exists in the codebase and works correctly, but is not documented anywhere. This causes users to:supportsImageInputorcapabilities.visionSolution
Added a new "Multimodal Support" section to the custom provider documentation that:
modalitiesproperty with a clear exampletext,audio,image,video,pdf)Changes
packages/web/src/content/docs/providers.mdxTesting
packages/opencode/src/provider/models.ts:53-58packages/opencode/src/provider/provider.ts:769-773packages/opencode/test/provider/provider.test.ts:1254-1257Example Configuration
{ "provider": { "myprovider": { "npm": "@ai-sdk/openai-compatible", "models": { "my-vision-model": { "name": "My Vision Model", "modalities": { "input": ["text", "image"], "output": ["text"] } } } } } }Related
@ai-sdk/openai-compatibletest/provider/provider.test.ts:1241-1280Impact
This documentation will: