LLM Provider Abstraction Layer #16
Description
Description:
Create abstract interface for supporting multiple LLM providers.
Requirements:
- Provider selection logic implemented
- Orchestration for sending messages through the currently selected provier
- registry pattern? benefits would be to have wiring to selected provider followed by execution immediately
- receiving a standardized context object with info about everything needed for LLM providers to work?
- Then each concrete implementations of LLM provider managers could have a validation data model of the Context to clearly control what must be present in the context for a selected API usage?
- Extension of LLM-related settings
- Authentication per provider in the settings
- Tool calling format standardized
- Streaming interface consistent
-
Discoverability of additional API functionality that some provider might allow but not others. #37--> Unclear at this point given that we don't know whether there will be any benefit to having Hatchling supporting the video/audio/images generation.-
Comprehensive way to expose what a provider can do (and is currently implemented in Hatchling) 'e.g. batches, image reading, function calling. Hos to tell the user at runtime what he can do when he uses this or that provider depending on what is integrated in Hatchling
-
That's a lot to think about.
Dependencies:
- Requires Issue Implement Modular Settings Architecture #12 for easy extension of settings