Add agent server hosting and invocation packages#45916
Merged
zhiyong-gayang merged 1 commit intoagentserver/invoke-reponsesfrom Mar 25, 2026
Merged
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
zhiyong-gayang
requested review from
JC-386,
lmazuel,
lusu-msft,
msyyc and
scbedd
as code owners
zhiyong-gayang
merged commit 5556989
into
agentserver/invoke-reponses
7 of 8 checks passed
pvaneck
pushed a commit
that referenced
this pull request
Apr 4, 2026
…5925) * create starlette package (#45754) - added type spec model generation - add model validator generation - creating a server * Lusu/response 0317 (#45757) * trying * generate contract models * add validator generator * fix model generation * add more unit tests * fix conflict * refined model generation * Lusu/response 0317 rename (#45764) * trying * generate contract models * add validator generator * fix model generation * add more unit tests * fix conflict * refined model generation * renamed the pacakge * response server apis (#45807) * create response * cancel and delete * fix options * remove old path * add samples * update pyproject * refining host * refined builders and hosting * refined package structur * remove unused code * add keep alive * fix minors * fix pytests * add file headers * add docstring for hosting * add conftest * add docstring * exclude scripts * add dev_requirement * fix build * fix build * fix generate models * fix pylint build * add ResponseIncompleteReason * refined stream * fix mypy * add ci * fix build * fix keyword * fix stream with background * ensure all contract cases covered * refined _rounting.py * fix stream payload * fix pylint * fix build * refining orchestrator * refining storing * add foundry storage provider * update response handler to decorator * remove dataclass * fix build * fix sphinx * try fix langchain-azure-ai version * Add agent server hosting and invocation packages (#45916) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Refactor responses package to leverage hosting for server hosting and route registration (#45921) Replace the standalone map_responses_server() function with a class-based ResponseHandler that plugs into AgentServer, following the same pattern as InvocationHandler. This enables multi-protocol composition, unified tracing via TracingHelper, automatic health probes, graceful shutdown delegation, and the x-platform-server identity header via hosting middleware. - Add ResponseHandler class with @create_handler decorator pattern - Integrate hosting's TracingHelper for OTel spans, W3C propagation, baggage - Fix Starlette 1.0.0 middleware compatibility in hosting _base.py - Add MultiProtocol sample showing both protocols on one AgentServer - Update all tests to use AgentServer + ResponseHandler pattern Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [agentserver-responses] refining response orchestration (#45923) * use handler docorator and refining orchestration * refining orchestration * refined orchestration * fixed conflict and handler registration * [agentserver][responses] fix replay stuck (#45930) * use handler docorator and refining orchestration * refining orchestration * refined orchestration * fixed conflict and handler registration * fix replay * Lusu/response refining (#45936) * fixed docstirng. refined ResponseContext * refined context * add payload helpers * use dict for contract * Lusu/response refining (#45944) * fixed docstirng. refined ResponseContext * refined context * add payload helpers * use dict for contract * fix inmemory * fix helper and sample * rename -hosting to -core (#45946) * rename AgentHost (#45948) * Remove redundant default_fetch_history_count_value from ResponsesServerOptions Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/eb08ee65-6dda-4937-ad91-45bb84a768cd Co-authored-by: lusu-msft <68949729+lusu-msft@users.noreply.github.com> * remove old -af and -lg from ci * fix multiprotocol sample * address PR #45925 review comments for -core and -invocations - Remove non-spec constants (AGENT_LOG_LEVEL, AGENT_GRACEFUL_SHUTDOWN_TIMEOUT) - Fix CHANGELOG: correct API names, add breaking changes section - Fix README env vars: add FOUNDRY_PROJECT_ARM_ID/SESSION_ID, remove non-spec - Wrap TracingHelper init in try/except for resilience - Add Python 3.14 classifier to pyproject.toml files - Move response header setting into try block for safety - Add input validation/sanitization for invocation_id and session_id - Add async function validation to handler decorators Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * add streaming and multi-turn invocation samples aligned to .NET - streaming_invoke_agent: SSE streaming with token-by-token output - multiturn_invoke_agent: session-based multi-turn conversations Aligns Python samples to the four .NET patterns: Sample1 (echo) -> simple_invoke_agent Sample2 (LRO) -> async_invoke_agent Sample3 (SSE) -> streaming_invoke_agent Sample4 (multi-turn) -> multiturn_invoke_agent Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * add FoundryEnrichmentSpanProcessor to enrich all spans with agent identity Register a span processor on the global TracerProvider so that gen_ai.agent.name, gen_ai.agent.version, gen_ai.agent.id, and microsoft.foundry.project.id are added to ALL spans — including those created by underlying frameworks (HTTP clients, LLM SDKs, etc.) — as required by the container image spec. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * resolving comments * [agentserver] remove nested methods in samples (#45977) * remove future import * remove nested funtion in samples * fix merge * fix apistub * update typespec and generated models * fix helper * remove output_message * rename health endpoint from /healthy to /readiness Per updated container spec, agentservice expects /readiness. Updated route, method name, docstrings, README, samples, and all tests across -core, -invocations, and -responses. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * update otel (#46003) * add _on_ending method to FoundryEnrichmentSpanProcessor The latest opentelemetry-sdk added _on_ending to the SpanProcessor interface, causing AttributeError at runtime. Add the required no-op method to fix compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Implement S-047 (x-agent-response-id header) and S-048 (agent_session_id resolution and stamping) Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/40114cbe-e583-4912-abc7-75c551a7e12c Co-authored-by: ankitbko <3169316+ankitbko@users.noreply.github.com> * Add unit tests for S-047 (Response ID Resolution) and S-048 (Session ID Resolution) Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/40114cbe-e583-4912-abc7-75c551a7e12c Co-authored-by: ankitbko <3169316+ankitbko@users.noreply.github.com> * Add CHANGELOG.md, set __version__, remove wildcard import, fix README private imports, simplify model access in samples Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/40114cbe-e583-4912-abc7-75c551a7e12c Co-authored-by: ankitbko <3169316+ankitbko@users.noreply.github.com> * Address code review feedback: clean up test, add comment for partition hint priority, add invalid header test Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/40114cbe-e583-4912-abc7-75c551a7e12c Co-authored-by: ankitbko <3169316+ankitbko@users.noreply.github.com> * [response] update validator and minor fix (#46029) * update otel * refined readme * updated model validator * fix model * deleted MAF and langchain adapter * fixed broken tests * Replace static-method classes with module-level functions and add tracing dupe protection Thread 17: Replace ErrorResponse.create() with create_error_response() and AgentLogger.get() with get_logger() module-level functions. Updated all call sites across core, invocations, and responses packages. Thread 18: Added Breaking Changes section to CHANGELOG.md documenting the API renames, removed constants, and health endpoint rename. Thread 21: Added module-level sentinel flags to _setup_trace_export, _setup_log_export, _setup_otlp_trace_export, and _setup_otlp_log_export to prevent duplicate exporters across multiple TracingHelper instantiations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Replace raw httpx with azure.core.AsyncPipelineClient in Foundry provider Migrated FoundryStorageProvider from raw httpx.AsyncClient to azure.core.AsyncPipelineClient, gaining built-in retry, logging, distributed tracing, and bearer-token authentication via AsyncBearerTokenCredentialPolicy. - Removed manual _auth_headers() — handled by pipeline policy - Removed http_client constructor parameter — pipeline is internal - Updated _foundry_errors.py to use azure.core.rest.HttpResponse - Removed httpx from pyproject.toml dependencies - Updated all unit tests to mock send_request instead of httpx methods Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Replace getattr(request, 'model', None) with request.model in README CreateResponse always has a model attribute, so the defensive getattr is unnecessary boilerplate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove azure-ai-agentserver-responses from this PR Responses package has been split into a separate PR on the agentserver/responses branch for independent review and iteration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Restore azure-ai-agentserver-agentframework and azure-ai-agentserver-langgraph from main These packages were inadvertently removed in an earlier commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix invoke_agent span parenting so framework child spans nest correctly (#46069) Add set_current_span/detach_context to TracingHelper so the invoke_agent span is attached as the active OTel context span. This ensures child spans created by framework handlers are parented under invoke_agent instead of appearing as siblings. - TracingHelper.set_current_span() attaches the span to context - TracingHelper.detach_context() restores the previous context - InvocationHandler propagates span_token through both streaming and non-streaming cleanup paths - Added test_span_parenting.py to verify parent-child relationship Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * added dev status inactive classifier * added log handler to root logger * Refactor AgentHost → AgentServerHost(Starlette) mixin inheritance Major architecture change per johanste's review: Core package: - Renamed AgentHost → AgentServerHost - Now inherits from Starlette directly (IS-A ASGI app) - Removed register_routes(), _build_app(), lazy app property - Routes, lifespan, and middleware set up in __init__ via super() - run()/run_async() use self as ASGI app Invocations package: - Renamed InvocationHandler → InvocationAgentServerHost - Now inherits from AgentServerHost - __init__ calls super() then extends self.routes - No more server parameter — this IS the server - Decorators are instance methods on the host User-facing API change: Before: server = AgentHost(); inv = InvocationHandler(server) After: app = InvocationAgentServerHost() Multi-protocol via diamond inheritance: class MyHost(InvocationHost, ResponsesHost): pass Updated all samples, READMEs, and tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Pass routes via super().__init__() instead of post-init mutation Routes are now collected by each mixin and passed up through super().__init__(routes=...) rather than appending to self.routes after construction. This avoids relying on Starlette's internal mutable routes list and ensures all routes are registered at Starlette initialization time. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Simplify _tracing.py: remove leaf_customer_span_id and baggage, fold methods Reduced _tracing.py from 1009 to 390 lines: - Removed leaf_customer_span_id baggage mechanism (_parse_baggage_key, _override_parent_span_id, _extract_context baggage override) - Removed set_baggage/detach_baggage/set_current_span/detach_context - Replaced start_request_span + manual context with request_span(end_on_exit=False) using start_as_current_span — child spans are correctly parented - Folded span_name, build_span_attrs, _prepare_request_span_args, span(), start_span() into a single request_span() context manager - TracingHelper public API is now: request_span, end_span, record_error, trace_stream - Updated invocations to use simplified tracing (removed baggage_token, span_token) - Removed baggage constants from InvocationConstants - Removed _parse_baggage_key tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix span parenting tests: use TestClient instead of ASGITransport Starlette's TestClient runs synchronously in the same thread context, so OTel ContextVar propagation works correctly. HTTPX's ASGITransport runs the ASGI app in a different async context where ContextVars don't propagate, causing span parenting to break in tests only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update sdk/agentserver/azure-ai-agentserver-core/azure/ai/agentserver/core/_base.py Co-authored-by: Johan Stenberg (MSFT) <johan.stenberg@microsoft.com> * Update sdk/agentserver/azure-ai-agentserver-invocations/samples/async_invoke_agent/async_invoke_agent.py Co-authored-by: Johan Stenberg (MSFT) <johan.stenberg@microsoft.com> * fixed broken appinsight env * addressed comments * Add get_logger to __all__ exports Since it is still `import`ed from azure-ai-agentserver-invocations, we need to keep it for now. * Fix up failing checks for azure-ai-agentserver-* --------- Co-authored-by: Zhiyong Yang <gary_yang2002@hotmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Zhiyong Yang <gayang@microsoft.com> Co-authored-by: ankitbko <3169316+ankitbko@users.noreply.github.com> Co-authored-by: Ankit Sinha <anksinha@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com> Co-authored-by: Ankit Sinha <ankitbko@gmail.com> Co-authored-by: Johan Stenberg (MSFT) <johan.stenberg@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
azure-ai-agentserver-hostingpackage — protocol-agnostic server infrastructure (AgentServer, tracing, graceful shutdown, health probes)azure-ai-agentserver-invocationspackage — invocation protocol handler (POST/GET/cancel endpoints, session IDs, OpenAPI spec)ci.ymlto include both new packagesTest plan
🤖 Generated with Claude Code