.NET Hosting: Cannot combine WithAITool with ChatClientAgentOptions

[MirkoMattioliSacmi]

Description

The Microsoft.Agents.AI.Hosting library has an architectural limitation that prevents using WithAITool() together with ChatClientAgentOptions features (such as AIContextProviderFactory, ChatMessageStoreFactory, etc.) when registering hosted agents.

Environment

• Microsoft.Agents.AI v1.0.0-preview.251002.1
• Microsoft.Agents.AI.Hosting v1.0.0-preview.251002.1
• .NET 10.0

Current Behavior

When using IHostApplicationBuilder.AddAIAgent(), developers must choose between:

1. Standard pattern - Can use WithAITool() but cannot set ChatClientAgentOptions
2. Factory delegate pattern - Can use ChatClientAgentOptions but tools registered via WithAITool() are lost

Expected Behavior

Developers should be able to set ChatClientAgentOptions and use WithAITool() to add tools.

Steps to Reproduce

Scenario 1: Try to use AIContextProviderFactory (fails)

// This approach allows using WithAITool()
var agentBuilder = builder.AddAIAgent("MyAgent", "You are helpful")
    .WithAITool(AIFunctionFactory.Create(MyTool.GetData));

// Problem: No way to configure AIContextProviderFactory or other ChatClientAgentOptions

Scenario 2: Use factory delegate (tools lost)

// This allows ChatClientAgentOptions but breaks tool registration
var agentBuilder = builder.AddAIAgent("MyAgent", (sp, name) =>
{
    var chatClient = sp.GetRequiredService<IChatClient>();
    
    return chatClient.CreateAIAgent(new ChatClientAgentOptions
    {
        Name = name,
        Instructions = "You are helpful",
        
        // Can use advanced options
        AIContextProviderFactory = context => new MyContextProvider(),
        
        // But tools registered via WithAITool() are NOT available here
        // The internal LocalAgentToolRegistry is not accessible from factory
    });
});

// This call is silently ignored
agentBuilder.WithAITool(AIFunctionFactory.Create(MyTool.GetData));

Root Cause

1. LocalAgentToolRegistry (used by WithAITool()) is internal and not accessible from factory delegates
2. No overload of AddAIAgent() exists that accepts both a factory delegate and supports subsequent WithAITool() calls
3. ChatClientAgentOptions can only be configured via factory delegate
4. Factory delegates bypass the tool registration mechanism entirely

Impact

This limitation affects any scenario requiring both custom tools AND advanced agent configuration:

• Custom context providers (user session data, language preferences, etc.)
• History summarization with custom message stores
• Any other ChatClientAgentOptions features

Proposed Solutions

Option 1: Expose tool registry to factory delegates

public static class ToolRegistryExtensions
{
    public static IReadOnlyList<AITool> GetRegisteredToolsForAgent(
        this IServiceProvider sp, 
        string agentName)
    {
        // Expose internal LocalAgentToolRegistry
    }
}

// Usage:
builder.AddAIAgent("MyAgent", (sp, name) =>
{
    var tools = sp.GetRegisteredToolsForAgent(name); // New API
    
    return chatClient.CreateAIAgent(new ChatClientAgentOptions
    {
        ChatOptions = new ChatOptions { Tools = [.. tools] },
        AIContextProviderFactory = context => new MyContextProvider()
    });
});
.WithAITool(AIFunctionFactory.Create(MyTool.GetData));

Option 2: Add new overload accepting options factory

public static IHostedAgentBuilder AddAIAgent(
    this IHostApplicationBuilder builder,
    string name,
    Func<IServiceProvider, string, ChatClientAgentOptions> optionsFactory)
{
    // Framework merges tools from WithAITool() into options
}

[markwallace-microsoft]

@DeagleGross could you take a look at this issue?