Skip to content

docs: add modalities configuration for custom providers#9898

Open
slycrel wants to merge 2 commits intoanomalyco:devfrom
slycrel:docs/add-modalities-documentation
Open

docs: add modalities configuration for custom providers#9898
slycrel wants to merge 2 commits intoanomalyco:devfrom
slycrel:docs/add-modalities-documentation

Conversation

@slycrel
Copy link

@slycrel slycrel commented Jan 21, 2026

docs: add modalities configuration for custom providers

Summary

Adds documentation for the modalities property that enables vision and multimodal support in custom OpenAI-compatible providers.

Problem

The modalities configuration property exists in the codebase and works correctly, but is not documented anywhere. This causes users to:

  • Believe vision/multimodal support is unavailable for custom providers
  • Try incorrect properties like supportsImageInput or capabilities.vision
  • Create duplicate issues asking how to enable image support
  • Potentially abandon OpenCode for other tools

Solution

Added a new "Multimodal Support" section to the custom provider documentation that:

  • Documents the modalities property with a clear example
  • Lists all supported input/output modalities (text, audio, image, video, pdf)
  • Explains validation behavior when modalities are not configured
  • Provides a practical example showing image input configuration

Changes

  • File: packages/web/src/content/docs/providers.mdx
  • Location: Added after the existing custom provider example, before the Troubleshooting section
  • Format: Follows the existing documentation style with code examples and tips

Testing

  • ✅ Configuration property exists in schema: packages/opencode/src/provider/models.ts:53-58
  • ✅ Used in capability detection: packages/opencode/src/provider/provider.ts:769-773
  • ✅ Covered by tests: packages/opencode/test/provider/provider.test.ts:1254-1257
  • ✅ Verified working with real-world custom provider (TaxHawk API)

Example Configuration

{
  "provider": {
    "myprovider": {
      "npm": "@ai-sdk/openai-compatible",
      "models": {
        "my-vision-model": {
          "name": "My Vision Model",
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          }
        }
      }
    }
  }
}

Related

Impact

This documentation will:

  • Reduce support burden from users asking about vision support
  • Improve discoverability of existing features
  • Help users configure multimodal providers correctly on first try
  • Prevent duplicate issues and confusion

- Document the 'modalities' property for enabling vision/multimodal support
- Add example showing image input configuration
- List all supported input/output modalities
- Explain validation behavior without modalities config

Fixes the discoverability issue where users cannot find how to enable
vision support for custom OpenAI-compatible providers.
@github-actions
Copy link
Contributor

The following comment was made by an LLM, it may be inaccurate:

Based on my search, I found one potentially related PR:

PR #5531: Feature/OpenAI compatible reasoning
#5531

This PR is mentioned in the current PR's description as relating to multimodal support for @ai-sdk/openai-compatible. It appears to be a complementary feature addition for the same provider ecosystem.

However, this is not a duplicate—it addresses a different feature (reasoning capabilities) rather than documentation of modalities configuration. The current PR #9898 is a documentation-only change that documents existing functionality.

No duplicate PRs found that are currently addressing the same documentation scope of modalities configuration for custom providers.

@slycrel
Copy link
Author

slycrel commented Jan 21, 2026

Related issue: #9897

@slycrel
Copy link
Author

slycrel commented Jan 21, 2026

Closes #9897

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant