Multi-Modal-Content-Safety-Evaluators#38002
Merged
w-javed merged 107 commits intoAzure:mainfrom Oct 28, 2024
Merged
Conversation
* Export the subscription data from the service connection * Update deploy-test-resources.yml --------- Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
…re#37709) * Update task_query_response.prompty remove required keys * Update task_simulate.prompty * Update task_query_response.prompty * Update task_simulate.prompty * Remove private variable and use kwargs * Add experimental tag to adv sim --------- Co-authored-by: Nagkumar Arkalgud <nagkumar@naarkalgworkmac.lan>
* Initial draft * Adding tests * Renaming parameter * Update container.py * Renaming test file * Fixing LINT issues * Update container.py * Update _base.py * Update _base.py * Fixing tests * Fixing tests * Adding support to disable response payload on write for AIO * Update CHANGELOG.md * Update _cosmos_client.py * Reacting to code review comments * Addressing code review feedback * Addressed CR feedback * Fixing pyLint errors * Fixing pylint errors * Update test_crud.py * Fixing svc regression * Update sdk/cosmos/azure-cosmos/azure/cosmos/aio/_container.py Co-authored-by: Anna Tisch <antisch@microsoft.com> * Reacting to code review feedback. * Update container.py * Update test_query_vector_similarity.py --------- Co-authored-by: Anna Tisch <antisch@microsoft.com>
* deprecate azure_germany * update * update * Update sdk/identity/azure-identity/azure/identity/_constants.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * update --------- Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com>
* Add default impl to handle token challenges * update version * update * update * update * update * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * update * Update sdk/core/azure-core/tests/test_utils.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * update --------- Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com>
…Evaluators (Azure#37707) * Make Credentials Required for Content Safety Evaluators * fix a typo * lint, fix content safety evaluator * revert test change * remove credential from rai_service
* Add getFeedRanges API * Add feedRange support in query changeFeed Co-authored-by: annie-mac <xinlian@microsoft.com>
…ure#37669) * during mindependency runs, dev_requirements on local relative paths are now checked for conflict with the targeted set of minimum dependencies * multiple type clarifications within azure-sdk-tools * added tests for new conflict resolution logic --------- Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
needuv
reviewed
Oct 25, 2024
...ai-evaluation/azure/ai/evaluation/_evaluators/_multimodal/_content_safety_multimodal_base.py
Show resolved
Hide resolved
needuv
reviewed
Oct 25, 2024
...ai-evaluation/azure/ai/evaluation/_evaluators/_multimodal/_content_safety_multimodal_base.py
Outdated
Show resolved
Hide resolved
needuv
reviewed
Oct 25, 2024
...zure-ai-evaluation/azure/ai/evaluation/_evaluators/_multimodal/_content_safety_multimodal.py
Outdated
Show resolved
Hide resolved
w-javed
changed the title
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/utils.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/utils.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluate/_utils.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
.../azure-ai-evaluation/azure/ai/evaluation/_evaluators/_content_safety/_content_safety_chat.py
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_adversarial_scenario.py
Outdated
Show resolved
Hide resolved
needuv
approved these changes
Oct 28, 2024
l0lawrence
pushed a commit
to l0lawrence/azure-sdk-for-python
that referenced
this pull request
Feb 19, 2025
* Initial-Commit-multimodal * Fix * Sync eng/common directory with azure-sdk-tools for PR 9092 (Azure#37713) * Export the subscription data from the service connection * Update deploy-test-resources.yml --------- Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Removing private parameter from __call__ of AdversarialSimulator (Azure#37709) * Update task_query_response.prompty remove required keys * Update task_simulate.prompty * Update task_query_response.prompty * Update task_simulate.prompty * Remove private variable and use kwargs * Add experimental tag to adv sim --------- Co-authored-by: Nagkumar Arkalgud <nagkumar@naarkalgworkmac.lan> * Enabling option to disable response payload on writes (Azure#37365) * Initial draft * Adding tests * Renaming parameter * Update container.py * Renaming test file * Fixing LINT issues * Update container.py * Update _base.py * Update _base.py * Fixing tests * Fixing tests * Adding support to disable response payload on write for AIO * Update CHANGELOG.md * Update _cosmos_client.py * Reacting to code review comments * Addressing code review feedback * Addressed CR feedback * Fixing pyLint errors * Fixing pylint errors * Update test_crud.py * Fixing svc regression * Update sdk/cosmos/azure-cosmos/azure/cosmos/aio/_container.py Co-authored-by: Anna Tisch <antisch@microsoft.com> * Reacting to code review feedback. * Update container.py * Update test_query_vector_similarity.py --------- Co-authored-by: Anna Tisch <antisch@microsoft.com> * deprecate azure_germany (Azure#37654) * deprecate azure_germany * update * update * Update sdk/identity/azure-identity/azure/identity/_constants.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * update --------- Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * Add default impl to handle token challenges (Azure#37652) * Add default impl to handle token challenges * update version * update * update * update * update * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * update * Update sdk/core/azure-core/tests/test_utils.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * update --------- Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> * Make Credentials Required for Content Safety and Protected Materials Evaluators (Azure#37707) * Make Credentials Required for Content Safety Evaluators * fix a typo * lint, fix content safety evaluator * revert test change * remove credential from rai_service * addFeedRangesAndUseFeedRangeInQueryChangeFeed (Azure#37687) * Add getFeedRanges API * Add feedRange support in query changeFeed Co-authored-by: annie-mac <xinlian@microsoft.com> * Update release date for core (Azure#37723) * Improvements to mindependency dev_requirement conflict resolution (Azure#37669) * during mindependency runs, dev_requirements on local relative paths are now checked for conflict with the targeted set of minimum dependencies * multiple type clarifications within azure-sdk-tools * added tests for new conflict resolution logic --------- Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com> * Need to add environment to subscription configuration (Azure#37726) Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com> * Enable samples for formrecognizer (Azure#37676) * multi-modal-changes * fixes * Fix with latest * dict-fix * adding-protected-material * adding-protected-material * adding-protected-material * bumping-version * adding assets * Added image in simulator * Added image in simulator * bumping-version * push-asset * assets * pushing asset * remove-containt-on-key * asset * asset2 * asset3 * asset4 * adding conftest * conftest * cred fix * asset-new * fix * asset * adding multi-modal-without-tests * asset-from-main * asset-from-main * fix * adding one test only * new asset * tests,fix: Sanitizer should replace with enum value not enum name * test-asset * [AutoRelease] t2-containerservicefleet-2024-09-24-42036(can only be merged by SDK owner) (Azure#37538) * code and test * Update CHANGELOG.md * update-testcase --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <119990644+ChenxiJiang333@users.noreply.github.com> Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com> * [AutoRelease] t2-dns-2024-09-25-81486(can only be merged by SDK owner) (Azure#37560) * code and test * update-testcase * Update CHANGELOG.md * Update test_mgmt_dns_test.py --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com> Co-authored-by: ChenxiJiang333 <119990644+ChenxiJiang333@users.noreply.github.com> * [AutoRelease] t2-appconfiguration-2024-10-09-68726(can only be merged by SDK owner) (Azure#37800) * code and test * update-testcase * Update pyproject.toml --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com> Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com> * code and test (Azure#37855) Co-authored-by: azure-sdk <PythonSdkPipelines> * [AutoRelease] t2-servicefabricmanagedclusters-2024-10-08-57405(can only be merged by SDK owner) (Azure#37768) * code and test * update-testcase * update-testcases --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com> * [AutoRelease] t2-containerinstance-2024-10-21-66631(can only be merged by SDK owner) (Azure#38005) * code and test * update-testcase * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com> Co-authored-by: ChenxiJiang333 <119990644+ChenxiJiang333@users.noreply.github.com> * [sdk generation pipeline] bump typespec-python 0.36.1 (Azure#38008) * update version * update package.json * [AutoRelease] t2-dnsresolver-2024-10-12-16936(can only be merged by SDK owner) (Azure#37864) * code and test * update-testcase * Update CHANGELOG.md * Update CHANGELOG.md --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com> Co-authored-by: ChenxiJiang333 <119990644+ChenxiJiang333@users.noreply.github.com> Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com> * new asset after fix in conftest * asset * chore: Update assets.json * Move perf pipelines to TME subscription (Azure#38020) Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com> * fix * after-comments * fix * asset * new asset with 1 test recording only * chore: Update assets.json * conftest fix * assets change * new test * few changes * removing proxy start * added all tests * asset * fixes * fixes with asset * asset-after-tax * enabling 2 more tests * unit test fix * asset * new asset * fixes per comments * changes by black * merge fix * pylint fix * pylint fix * ground test fix * fixes - pylint, black, mypy * more tests * docstring fixes * doc string fix * asset * few updates after Nagkumar review --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Nagkumar Arkalgud <nagkumar91@users.noreply.github.com> Co-authored-by: Nagkumar Arkalgud <nagkumar@naarkalgworkmac.lan> Co-authored-by: Fabian Meiswinkel <fabianm@microsoft.com> Co-authored-by: Anna Tisch <antisch@microsoft.com> Co-authored-by: Xiang Yan <xiangsjtu@gmail.com> Co-authored-by: Paul Van Eck <paulvaneck@microsoft.com> Co-authored-by: Neehar Duvvuri <40341266+needuv@users.noreply.github.com> Co-authored-by: Annie Liang <64233642+xinlian12@users.noreply.github.com> Co-authored-by: annie-mac <xinlian@microsoft.com> Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com> Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com> Co-authored-by: kdestin <101366538+kdestin@users.noreply.github.com> Co-authored-by: ChenxiJiang333 <119990644+ChenxiJiang333@users.noreply.github.com> Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com> Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We are adding new set of evaluators under Content Safety for multi modal use cases supporting images along with text.