Skip to content

fix: use locally bundled models when remote_models disabled#11691

Closed
sayan-oai wants to merge 1 commit intodev/sayan/warn-on-fallback-metadatafrom
dev/sayan/bundled-models-when-remote-models-disabled
Closed

fix: use locally bundled models when remote_models disabled#11691
sayan-oai wants to merge 1 commit intodev/sayan/warn-on-fallback-metadatafrom
dev/sayan/bundled-models-when-remote-models-disabled

Conversation

@sayan-oai
Copy link
Collaborator

Why

We see issues like #11527 when a user has remote_models disabled, as we always use the default fallback ModelInfo. This causes issues with model performance. #11690 helps by warning the user when they are using the default fallback, but we shouldn't use the fallback if we have matching ModelInfo in the locally bundled models.json.

I added ModelsManager::bundled_models as a new field, but we could also read ModelsManager::remote_models if we are ok with using cached model info when remote_models is disabled.

Tests

Updated existing test to confirm we don't use the default model info when remote_models is disabled with a slug that's in the locally bundled models.json.

@sayan-oai sayan-oai closed this Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant