fix: clean optimizer tests — proper patch() + Azure Entra ID integration#10
Merged
fix: clean optimizer tests — proper patch() + Azure Entra ID integration#10
Conversation
- Rewrite test_optimizer.py using patch('...PydanticAgent', ...) cleanly
- Add TestAzureEntraModel class (3 tests covering default gpt-5.2-chat deployment)
- Update test_optimizer_integration.py to use azure_entra_model() (gpt-5.2-chat)
instead of OPENAI_API_KEY skip guard -- all 3 now pass against real Azure
- Verified full test->optimize->test loop end-to-end: 3/3 passed in 64s
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
…nstead pytest-aitest already provides build_model_from_string() which handles Azure Entra ID auth automatically from 'azure/gpt-5.2-chat' style strings. - Delete azure_entra_model() from optimizer.py and __init__.py (duplication) - optimize_instruction() now uses build_model_from_string() internally - Default model changed to 'azure/gpt-5.2-chat' - Model strings use provider/model format (same as pytest-aitest Provider) - Remove TestAzureEntraModel unit tests (testing dead code) - Update integration tests: no extra import needed - 3/3 integration tests pass against azure/gpt-5.2-chat via Entra ID Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds test_full_optimize_loop which validates the hero use case end-to-end:
1. Run Copilot with weak instruction ('no comments or documentation needed')
-> code produced has no docstrings (verified)
2. Call optimize_instruction() with failing criterion
-> optimizer identifies the conflict, suggests explicit docstring mandate
3. Run Copilot again with improved instruction
-> code now contains Google-style docstrings (verified)
Result: Docstrings round 1: False, round 2: True (passed in 36s)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…equirement
Unit tests were failing in CI with 'AZURE_API_BASE or AZURE_OPENAI_ENDPOINT
required' because build_model_from_string('azure/gpt-5.2-chat') runs before
PydanticAgent is mocked. Now both are patched together.
Also strengthens test_uses_default_model: verifies the exact model string
passed to build_model_from_string, and that the resolved model is forwarded.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sbroenne
pushed a commit
that referenced
this pull request
Feb 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Why
The old \sys.modules\ patching mutated live modules with no cleanup, polluting other tests. The correct target is now \patch('...optimizer.PydanticAgent', ...)\ since pydantic-ai is a top-level import.
Verified