Python: Add GeminiChatClient by holtvogt · Pull Request #4847 · microsoft/agent-framework

holtvogt · 2026-03-23T07:34:52Z

Motivation and Context

Google Gemini is one of the leading LLM providers and is not yet supported as a first-class integration in Agent Framework. This PR adds a dedicated agent-framework-gemini package that gives developers a native, idiomatic way to use Google Gemini models within Agent Framework.

Description

This PR introduces the agent-framework-gemini package. A new provider integration built on the official google-genai SDK (>=1.0.0,<2.0.0).

Google API Reference

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the Contribution Guidelines
All unit tests pass, and I have added new tests where possible
Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Reference

Addresses Gemini client #39

Copilot

Pull request overview

Adds a first-class Google Gemini provider integration to the Python Agent Framework workspace via a new agent-framework-gemini package (built on google-genai), plus runnable samples and tests.

Changes:

Introduces agent_framework_gemini.GeminiChatClient with Gemini-specific GeminiChatOptions/ThinkingConfig and message/tool mapping.
Adds unit + integration tests for text, streaming, tools, thinking config, grounding, code execution, and structured output.
Adds Google Gemini sample scripts and wires the new package/dependency into the Python workspace (uv.lock, .env.example).

Reviewed changes

Copilot reviewed 15 out of 18 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
python/uv.lock	Adds `agent-framework-gemini` workspace member and locks `google-genai` dependency.
python/.env.example	Documents `GEMINI_API_KEY` / `GEMINI_CHAT_MODEL_ID` env vars.
python/packages/gemini/pyproject.toml	Defines the new distributable package and its dependencies/tooling config.
python/packages/gemini/agent_framework_gemini/_chat_client.py	Implements the Gemini chat client (streaming, tools, config mapping, response parsing).
python/packages/gemini/agent_framework_gemini/init.py	Exposes the package public API + version.
python/packages/gemini/tests/test_chat_client.py	Adds unit tests for conversion/config/streaming behavior.
python/packages/gemini/tests/test_chat_client_integration.py	Adds integration tests gated by `GEMINI_API_KEY`.
python/samples/02-agents/providers/google/*.py	Adds runnable Gemini provider samples (basic, advanced thinking, grounding, code execution).
python/samples/02-agents/providers/google/README.md	Documents the new Gemini samples.
python/packages/gemini/README.md / LICENSE / AGENTS.md / py.typed	Packaging docs/metadata for the new provider.

eavanvalkenburg

Automated Code Review

Reviewers: 4 | Confidence: 89%

✓ Correctness

This PR adds a well-structured Google Gemini integration package following the established patterns of other providers (Anthropic, Ollama, Bedrock). The implementation correctly handles message conversion, tool calling, streaming, system instructions, and finish reason mapping. The code is defensively written with proper edge-case handling (missing function call IDs, unmatched function results, consecutive tool messages). No blocking correctness issues were found. One minor suggestion relates to passing response_format through to the stream finalizer for Pydantic-model-based structured output, consistent with how Anthropic and Ollama do it, though the Gemini integration primarily uses response_schema as a dict so this is low-impact.

✗ Security Reliability

This PR adds a new Google Gemini chat client integration. The implementation follows established patterns for settings, API key handling (SecretString), and message conversion. However, there is one reliability issue: the class inheritance (MRO) order for GeminiChatClient swaps ChatMiddlewareLayer and FunctionInvocationLayer compared to every other provider in the codebase, which could cause middleware to not properly wrap function invocations. No secrets are hardcoded, JSON parsing is safely wrapped, and input validation is present at key trust boundaries.

✓ Test Coverage

The new Gemini package has a solid unit test suite (879 lines) covering the main paths: text responses, streaming, tool calling, config mapping, message conversion, finish reasons, thinking config, and built-in tools. However, there are notable test coverage gaps: no test for empty/None candidates in API responses, no test for the streaming get_final_response() finalization path, no test for _coerce_to_dict with a JSON string that parses to a non-dict type (e.g., JSON array), and no test for response_schema set without response_format. These are edge cases that the implementation appears to handle correctly, but without tests they could regress silently.

✗ Design Approach

The Gemini integration is well-structured and follows established patterns from other providers. The main design concern is that _prepare_gemini_messages resolves the required name field for Gemini's FunctionResponse by scanning prior conversation history to build a call_id → name map. The root cause is that Content.from_function_result in the framework does not persist the function name (confirmed: the name field is always None on function_result Contents as created by _execute_function_call). The history-scanning workaround works for the common case but silently drops tool results whenever messages are trimmed — a realistic production scenario (context-window management, memory layers). A straightforward fix exists at the framework level: add a name parameter to Content.from_function_result and pass it from _execute_function_call, then consume content.name directly in the Gemini client. One secondary concern: setting response_schema alone (without response_format) silently enables application/json MIME type, which is undocumented behavior in the option's docstring.

Automated review by eavanvalkenburg's agents

markwallace-microsoft · 2026-03-23T08:48:30Z

Python Test Coverage Report •

File	Stmts	Miss	Cover
packages/gemini/agent_framework_gemini
_chat_client.py	280	0	100%
TOTAL	27534	3198	88%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
5570	20 💤	0 ❌	0 🔥	1m 34s ⏱️

eavanvalkenburg · 2026-03-23T08:51:06Z

@holtvogt thanks for looking into this, I've added some comments

Also updates constructor parameter order for environment file handling

eavanvalkenburg

we're getting close, some more detailed things left

holtvogt · 2026-04-10T15:20:18Z

@eavanvalkenburg The tool protocol implementation should be fixed now. The failing tests were caused by using model_id in ChatResponse. I updated it, and the tests are running 💚 Returning it back to you for another look 👀

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>

holtvogt · 2026-04-10T16:25:53Z

@eavanvalkenburg Sending this back like a well-traveled boomerang. Ready for another look! 🏓

Now reflects new options and built-in tool factory methods

eavanvalkenburg

Small comment on the build system, otherwise good to go! Thanks

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>

holtvogt · 2026-04-14T09:39:08Z

@eavanvalkenburg Updated the build system. Can you run the tests again?

The Gemini chat client references several `google.genai.types` symbols (`FileSearch`, `ThinkingLevel`, `SearchTypes`, `McpServer`, `StreamableHttpTransport`, plus call-site keyword args `mcp_servers` and `search_types`) that are not present at the lower bound of `google-genai>=1.0.0`. At `lowest-direct` resolution this caused `validate-dependency-bounds-test` to fail for `packages/gemini` with eleven `reportAttributeAccessIssue` / `reportUnknownVariableType` errors. Walking the upstream `google.genai.types` API: - `GoogleMaps`, `AuthConfig`: present from 1.40.0 - `FileSearch`: introduced in 1.49.0 - `ThinkingLevel`: introduced in 1.55.0 - `SearchTypes`, `McpServer`, `StreamableHttpTransport`: introduced in 1.65.0 Bump the lower bound to 1.65.0 — the minimum version that exposes every symbol the package actually uses. Keep the `<2.0.0` upper cap unchanged. With this bump `validate-dependency-bounds-test` passes for both lower and upper resolution scenarios across all 27 workspace packages. Latent since microsoft#4847 (Gemini package introduction in 1.1.0); aggravated by subsequent feature additions that pulled in newer `types.*` symbols.

* Python: bump package versions for 1.2.1 release PATCH bump (1.2.0 -> 1.2.1) for the released cohort. The release window covers two PRs, no new public APIs: - agent-framework-core: prevent inner_exception from being lost in AgentFrameworkException (#5167) - samples: add requirements.txt and .env.example to the a2a/ hosting sample for pip-based setup (#5510) Per lockstep convention, all 21 beta packages stamp 1.0.0b260428 and all 3 alpha packages stamp 1.0.0a260428, regardless of per-package code churn. Every non-core package floor on agent-framework-core is raised to >=1.2.1 to keep cohort signaling consistent. Date stamp reflects the local (Asia) cut date 2026-04-28. * Python: silence pyright unknown-type warnings in hosted-env detection `azure.ai.agentserver.core` is probed at runtime via `importlib.util.find_spec` and is not a declared dependency. The existing `# pyright: ignore[reportMissingImports]` suppresses the missing-import warning, but at `lowest-direct` resolution pyright still reports the imported symbol (`AgentConfig`) and its members (`from_env`, `is_hosted`) as unknown, breaking `validate-dependency-bounds-test` for `packages/core`. Extend the existing ignore to cover `reportUnknownVariableType` on the import and `reportUnknownMemberType` on the call site so the bounds check returns to green. Behavior is unchanged. Latent since #5455 (shipped in 1.2.0). * Python: raise agent-framework-gemini lower bound to google-genai>=1.65.0 The Gemini chat client references several `google.genai.types` symbols (`FileSearch`, `ThinkingLevel`, `SearchTypes`, `McpServer`, `StreamableHttpTransport`, plus call-site keyword args `mcp_servers` and `search_types`) that are not present at the lower bound of `google-genai>=1.0.0`. At `lowest-direct` resolution this caused `validate-dependency-bounds-test` to fail for `packages/gemini` with eleven `reportAttributeAccessIssue` / `reportUnknownVariableType` errors. Walking the upstream `google.genai.types` API: - `GoogleMaps`, `AuthConfig`: present from 1.40.0 - `FileSearch`: introduced in 1.49.0 - `ThinkingLevel`: introduced in 1.55.0 - `SearchTypes`, `McpServer`, `StreamableHttpTransport`: introduced in 1.65.0 Bump the lower bound to 1.65.0 — the minimum version that exposes every symbol the package actually uses. Keep the `<2.0.0` upper cap unchanged. With this bump `validate-dependency-bounds-test` passes for both lower and upper resolution scenarios across all 27 workspace packages. Latent since #4847 (Gemini package introduction in 1.1.0); aggravated by subsequent feature additions that pulled in newer `types.*` symbols. * Python: add dependabot bumps to 1.2.1 CHANGELOG Catalog the 15 dependabot dependency updates that merged on `upstream/main` between python-1.2.0 and the 1.2.1 cut window under a new Changed section: - Workspace dev/runtime deps: `rich`, `prek`, `python-multipart`, `pyasn1`, `pytest` (ag-ui, devui, lab), `uv` (lab) - Frontend deps: `vite` (devui, chatkit), `postcss` (devui, chatkit, handoff), `picomatch` (devui, handoff) CHANGELOG-only — no source or pyproject.toml changes. PRs themselves merged upstream independently of this release branch and will be brought in via the PR merge.

holtvogt added 8 commits March 23, 2026 08:21

Add agent-framework-gemini package

0e35f04

Add AGENTS.md documentation

33f437f

Add LICENSE file

38d3d28

Add README.md for agent-framework-gemini package

29ed60c

Add Google Gemini API keys to .env.example

0dace12

Add Google Gemini chat client implementation

06de5cf

Add tests for GeminiChatClient

24fdbeb

Add Google Gemini agent examples

dbc1709

Copilot AI review requested due to automatic review settings March 23, 2026 07:34

markwallace-microsoft added documentation Improvements or additions to documentation python labels Mar 23, 2026

Copilot started reviewing on behalf of holtvogt March 23, 2026 07:35 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

eavanvalkenburg reviewed Mar 23, 2026

View reviewed changes

Comment thread python/packages/gemini/agent_framework_gemini/_chat_client.py

Comment thread python/packages/gemini/tests/test_gemini_client.py

Comment thread python/packages/gemini/tests/test_gemini_client.py

Comment thread python/packages/gemini/tests/test_gemini_client.py

holtvogt added 14 commits March 23, 2026 17:42

Merge branch 'main' into feat/add-gemini-client

41e7f6e

Fix client inheritence order

85a86a1

Update Gemini agent examples

db6521b

Update documentation

54af10b

Update AGENTS.md

cf4a6fb

Add tests for JSON string handling in GeminiChatClient

0b0afd1

Add final response assembly test in GeminiChatClient

8964929

Add tests for handling empty candidates in GeminiChatClient

533aa7c

Improve Pydantic response handling in GeminiChatClient

308c474

Add tests for function result resolution and callable tool normalization

80580fc

Add test for function result resolution when call_id is generated

aeda903

Refactor GeminiChatClient to correct inheritance order

ff52460

Also updates constructor parameter order for environment file handling

Enhance documentation and clarify Gemini-specific fields

c7529f9

Update ThinkingConfig with new attributes and type

5a839e6

Merge branch 'main' into feat/add-gemini-client

57df672

eavanvalkenburg reviewed Apr 10, 2026

View reviewed changes

holtvogt and others added 2 commits April 10, 2026 17:25

Unify Gemini model name

313a001

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>

Update Agent Framework core version

29855dd

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>

eavanvalkenburg self-assigned this Apr 10, 2026

eavanvalkenburg added this to Agent Framework Apr 10, 2026

eavanvalkenburg moved this to Community PR in Agent Framework Apr 10, 2026

holtvogt added 4 commits April 10, 2026 17:27

Add Python 3.14 in classifiers

149b463

Replace kwargs with parameters in tool factories

e7e4183

Refactor chat options handling in Gemini client

889a846

Add tests for handling unknown and consumed keys

fa18e0e

holtvogt requested a review from eavanvalkenburg April 10, 2026 16:26

Update Gemini documentation

d29c077

Now reflects new options and built-in tool factory methods

eavanvalkenburg reviewed Apr 14, 2026

View reviewed changes

Comment thread python/packages/gemini/pyproject.toml Outdated

holtvogt and others added 2 commits April 14, 2026 11:04

Change build system to flit

2c44e29

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>

Merge branch 'main' into feat/add-gemini-client

e7d5618

eavanvalkenburg reviewed Apr 14, 2026

View reviewed changes

Comment thread python/packages/gemini/pyproject.toml Outdated

holtvogt added 2 commits April 14, 2026 11:37

Fix build system in pyproject.toml

b05b627

Fix type checking for generate_content_stream

8fac9e4

eavanvalkenburg approved these changes Apr 14, 2026

View reviewed changes

moonbox3 approved these changes Apr 14, 2026

View reviewed changes

Comment thread python/packages/gemini/pyproject.toml

eavanvalkenburg added this pull request to the merge queue Apr 14, 2026

Merged via the queue into microsoft:main with commit 485af07 Apr 14, 2026
32 checks passed

github-project-automation Bot moved this from Community PR to Done in Agent Framework Apr 14, 2026

Conversation

holtvogt commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation and Context

Description

Google API Reference

Contribution Checklist

Reference

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eavanvalkenburg left a comment

Choose a reason for hiding this comment

Automated Code Review

✓ Correctness

✗ Security Reliability

✓ Test Coverage

✗ Design Approach

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markwallace-microsoft commented Mar 23, 2026 • edited by moonbox3 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Python Unit Test Overview

Uh oh!

eavanvalkenburg commented Mar 23, 2026

Uh oh!

eavanvalkenburg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

holtvogt commented Apr 10, 2026

Uh oh!

holtvogt commented Apr 10, 2026

Uh oh!

eavanvalkenburg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

holtvogt commented Apr 14, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

holtvogt commented Mar 23, 2026 •

edited

Loading

markwallace-microsoft commented Mar 23, 2026 •

edited by moonbox3

Loading