Conversation
- Updated the API version for job definition commands: start-job, stop-job, update, and wait. - Enhanced job definition update command with new properties: connections, data integrity validation, preserve permissions, and schedule. - Updated job run commands: list, show, with new properties for scheduled execution time and warnings. - Updated project commands: create, delete, list, show, and update with new API version and improved descriptions. - Added support for S3 with HMAC in endpoint commands: create and update. - Bumped package version to 1.3.0.
- Added test_storage_mover_connection_scenarios for new connection resource CRUD - Added test_storage_mover_endpoint_s3_with_hmac_scenarios for S3WithHmac endpoint - Added test_storage_mover_job_definition_schedule_scenarios for schedule, preserve-permissions, data-integrity-validation - Removed 7 stale test recordings from 2025-07-01 API version
Update source_type enum to match 2025-12-01 API spec: - Added: ALIBABA, DELL_EMC, OTHER - Removed: BACKBLAZE, CLOUDFLARE - Updated test to use MINIO instead of removed BACKBLAZE - Updated HISTORY.rst changelog Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Validation for Breaking Change Starting...
Thanks for your contribution! |
|
Hi @sssash18, |
|
Thank you for your contribution! We will review the pull request and get back to you soon. |
|
The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR. Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions). pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>
|
CodeGen Tools Feedback CollectionThank you for using our CodeGen tool. We value your feedback, and we would like to know how we can improve our product. Please take a few minutes to fill our codegen survey |
|
There was a problem hiding this comment.
Pull request overview
This PR updates the storage-mover Azure CLI extension to align with the 2025-12-01 GA API version, introducing new resource types/parameters and expanding scenario coverage to exercise the new surface area.
Changes:
- Bump extension version to 1.3.0 and document 2025-12-01 GA changes in
HISTORY.rst. - Add/refresh command surface for connections, S3WithHMAC endpoints, endpointKind, and job definition schedule/connections/preserve-permissions/data-integrity-validation.
- Update scenario tests + recordings to cover new parameters and API version.
Reviewed changes
Copilot reviewed 64 out of 66 changed files in this pull request and generated 19 comments.
Show a summary per file
| File | Description |
|---|---|
| src/storage-mover/setup.py | Bumps extension version to 1.3.0. |
| src/storage-mover/HISTORY.rst | Adds 1.3.0 changelog for 2025-12-01 GA API features. |
| src/storage-mover/azext_storage_mover/tests/latest/test_storage_mover.py | Updates/extends scenario tests for endpointKind, cloud-to-cloud job definitions, schedules, connections, and S3WithHMAC endpoints. |
| src/storage-mover/azext_storage_mover/tests/latest/recordings/test_storage_mover_endpoint_multi_cloud_connector_scenarios.yaml | Updates recording to 2025-12-01 and includes endpointKind/no-kind behavior. |
| src/storage-mover/azext_storage_mover/tests/latest/recordings/test_storage_mover_agent_upload_limit_schedule_scenarios.yaml | Updates recording API version to 2025-12-01. |
| src/storage-mover/azext_storage_mover/custom.py | Adds endpoint-kind + S3WithHMAC custom create/update plumbing and refactors endpoint arg shapes. |
| src/storage-mover/azext_storage_mover/commands.py | Registers new custom commands for S3WithHMAC create/update. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/project/_wait.py | Switches to 2025-12-01 API and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/project/_update.py | Switches to 2025-12-01 API and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/project/_show.py | Switches to 2025-12-01 API and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/project/_list.py | Switches to 2025-12-01 API and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/project/_delete.py | Switches to 2025-12-01 API and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/project/_create.py | Switches to 2025-12-01 API and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/project/__cmd_group.py | Updates command group help text. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_run/_show.py | Switches to 2025-12-01 and adds scheduledExecutionTime/triggerType/warnings fields. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_run/_list.py | Switches to 2025-12-01 and adds scheduledExecutionTime/triggerType/warnings fields. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_run/__cmd_group.py | Updates command group help text. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/_wait.py | Switches to 2025-12-01 and extends schema with new job definition properties. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/_update.py | Switches to 2025-12-01 and adds new job definition args (schedule, connections, preserve-permissions, data-integrity-validation). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/_stop_job.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/_start_job.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/_show.py | Switches to 2025-12-01 and extends schema with new job definition properties. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/_list.py | Switches to 2025-12-01 and extends schema with new job definition properties. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/_delete.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/_create.py | Switches to 2025-12-01 and adds new job definition args (schedule, connections, preserve-permissions, data-integrity-validation). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/job_definition/__cmd_group.py | Updates command group help text. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/identity/_show.py | Switches to 2025-12-01 and extends endpoint schema with endpointKind + S3WithHMAC discrimination. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/identity/_remove.py | Switches to 2025-12-01 and extends endpoint schema with endpointKind + S3WithHMAC discrimination. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/identity/_assign.py | Switches to 2025-12-01 and extends endpoint schema with endpointKind + S3WithHMAC discrimination. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/_wait.py | Switches to 2025-12-01 and extends endpoint schema with endpointKind + S3WithHMAC discrimination. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/_update.py | Switches to 2025-12-01 and updates endpoint update surface (drops blob-container update args). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/_show.py | Switches to 2025-12-01 and extends endpoint schema with endpointKind + S3WithHMAC discrimination. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/_list.py | Switches to 2025-12-01 and extends endpoint schema with endpointKind + S3WithHMAC discrimination. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/_delete.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/_create.py | Switches to 2025-12-01, adds endpointKind and S3WithHMAC modeling, and renames blob-container arg object. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/endpoint/__cmd_group.py | Updates command group help text. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/connection/_wait.py | Adds new AAZ “connection wait” command (2025-12-01). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/connection/_update.py | Adds new AAZ “connection update” command (2025-12-01). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/connection/_show.py | Adds new AAZ “connection show” command (2025-12-01). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/connection/_list.py | Adds new AAZ “connection list” command (2025-12-01). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/connection/_delete.py | Adds new AAZ “connection delete” command (2025-12-01). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/connection/_create.py | Adds new AAZ “connection create” command (2025-12-01). |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/connection/init.py | Exposes the new connection command group modules. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/connection/__cmd_group.py | Adds new “storage-mover connection” command group. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/agent/_wait.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/agent/_update.py | Switches to 2025-12-01 and updates help/examples. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/agent/_unregister.py | Switches to 2025-12-01 and changes confirmation/help text. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/agent/_show.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/agent/_list.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/agent/_create.py | Switches to 2025-12-01 and adds missing command registration + name validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/agent/__cmd_group.py | Updates command group help text. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/_wait.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/_update.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/_show.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/_list.py | Switches to 2025-12-01 API version for subscription/RG list variants. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/_delete.py | Switches to 2025-12-01 and changes confirmation/help text. |
| src/storage-mover/azext_storage_mover/aaz/latest/storage_mover/_create.py | Switches to 2025-12-01 and adds name format validation. |
| src/storage-mover/azext_storage_mover/_params.py | Adds endpoint-kind and S3WithHMAC argument wiring + updated enums. |
| src/storage-mover/azext_storage_mover/_help.py | Adds help entries for S3WithHMAC endpoint custom commands. |
| def _get_args_for_endpoint_for_storage_container(endpoint_name, resource_group, storage_mover_name, | ||
| blob_container_name=None, storage_account_resource_id=None, | ||
| description=None): | ||
| endpoint_kind=None, description=None): | ||
| args = { | ||
| "endpoint_name": endpoint_name, "resource_group": resource_group, "storage_mover_name": storage_mover_name, | ||
| "description": description, | ||
| "storage_blob_container": { | ||
| "endpoint_kind": endpoint_kind, | ||
| "azure_storage_blob_container": { | ||
| "blob_container_name": blob_container_name, | ||
| "storage_account_resource_id": storage_account_resource_id, | ||
| } |
There was a problem hiding this comment.
_get_args_for_endpoint_for_storage_container now unconditionally adds endpoint_kind and an azure_storage_blob_container object to the args dict. This helper is also used by update flows (e.g., endpoint_update_for_storage_container calls it with only description), but the generated AAZ storage-mover endpoint update command schema no longer defines endpoint_kind/azure_storage_blob_container. This mismatch is likely to break the custom update commands at runtime with unrecognized/invalid args. Consider splitting create vs update arg builders (update should only send supported fields like description), or updating the AAZ update schema to accept these fields.
| def _get_args_for_endpoint_for_smb(endpoint_name, resource_group, storage_mover_name, | ||
| host=None, share_name=None, username_uri=None, password_uri=None, description=None): | ||
| host=None, share_name=None, username_uri=None, password_uri=None, | ||
| endpoint_kind=None, description=None): | ||
| args = { | ||
| "endpoint_name": endpoint_name, "resource_group": resource_group, "storage_mover_name": storage_mover_name, | ||
| "description": description, | ||
| "endpoint_kind": endpoint_kind, | ||
| "smb_mount": { |
There was a problem hiding this comment.
Several endpoint arg helpers (e.g., _get_args_for_endpoint_for_smb) now always include endpoint_kind and endpoint-type-specific objects in the args dict. The generated AAZ storage-mover endpoint update command only supports updating a limited set of endpoint-type payloads, so passing these extra keys (even as None) can cause update commands (like endpoint_update_for_smb / endpoint_update_for_nfs) to fail argument validation. Consider omitting endpoint_kind and endpoint-type objects for update calls unless the AAZ update schema explicitly supports them.
| s3_with_hmac.source_type = AAZStrArg( | ||
| options=["source-type"], | ||
| help="The source type of S3WithHmac endpoint.", | ||
| enum={"BACKBLAZE": "BACKBLAZE", "CLOUDFLARE": "CLOUDFLARE", "GCS": "GCS", "IBM": "IBM", "MINIO": "MINIO"}, |
There was a problem hiding this comment.
The generated s3_with_hmac.source_type enum still lists deprecated values (e.g., BACKBLAZE, CLOUDFLARE) and does not include the new 2025-12-01 values mentioned in HISTORY/_params.py (e.g., ALIBABA, DELL_EMC, OTHER). This makes az storage-mover endpoint create --s3-with-hmac ... --source-type ... accept/reject the wrong values relative to the API spec.
| enum={"BACKBLAZE": "BACKBLAZE", "CLOUDFLARE": "CLOUDFLARE", "GCS": "GCS", "IBM": "IBM", "MINIO": "MINIO"}, | |
| enum={"ALIBABA": "ALIBABA", "DELL_EMC": "DELL_EMC", "GCS": "GCS", "IBM": "IBM", "MINIO": "MINIO", "OTHER": "OTHER"}, |
| help="The Azure Key Vault secret URI which stores the username. Use empty string to clean-up existing value.", | ||
| ) | ||
| credentials.secret_key_uri = AAZStrArg( | ||
| options=["secret-key-uri"], | ||
| help="The Azure Key Vault secret URI which stores the password. Use empty string to clean-up existing value.", |
There was a problem hiding this comment.
In the S3-with-HMAC credentials args, the help strings say the Key Vault secrets store “username”/“password”, but these are access key / secret key URIs. Please update the help text to match the actual semantics to avoid confusing users and producing incorrect CLI docs.
| help="The Azure Key Vault secret URI which stores the username. Use empty string to clean-up existing value.", | |
| ) | |
| credentials.secret_key_uri = AAZStrArg( | |
| options=["secret-key-uri"], | |
| help="The Azure Key Vault secret URI which stores the password. Use empty string to clean-up existing value.", | |
| help="The Azure Key Vault secret URI which stores the access key. Use empty string to clean-up existing value.", | |
| ) | |
| credentials.secret_key_uri = AAZStrArg( | |
| options=["secret-key-uri"], | |
| help="The Azure Key Vault secret URI which stores the secret key. Use empty string to clean-up existing value.", |
| - name: endpoint create-for-s3-with-hmac | ||
| text: > | ||
| az storage-mover endpoint create-for-s3-with-hmac -g "rg" --storage-mover-name "mover_name" | ||
| -n "endpoint_s3" --source-uri "https://s3.example.com/bucket" --source-type BACKBLAZE |
There was a problem hiding this comment.
The help example for storage-mover endpoint create-for-s3-with-hmac uses --source-type BACKBLAZE, but the extension’s argument definition (_params.py) and HISTORY indicate BACKBLAZE was removed for the 2025-12-01 API version. Update the example to use a currently supported value (e.g., MINIO, ALIBABA, etc.) so az help doesn’t suggest invalid input.
| -n "endpoint_s3" --source-uri "https://s3.example.com/bucket" --source-type BACKBLAZE | |
| -n "endpoint_s3" --source-uri "https://s3.example.com/bucket" --source-type MINIO |
| required=True, | ||
| id_part="child_name_1", | ||
| fmt=AAZStrArgFormat( | ||
| pattern="^[A-Za-z0-9][A-Za-z0-9_-]{0,20}", |
There was a problem hiding this comment.
The AAZStrArgFormat.pattern for connection_name is missing the $ end anchor, which weakens the validation of the intended max length. Please add $ so the entire input must match (e.g., ...{0,20}$).
| pattern="^[A-Za-z0-9][A-Za-z0-9_-]{0,20}", | |
| pattern="^[A-Za-z0-9][A-Za-z0-9_-]{0,20}$", |
| @register_command( | ||
| "storage-mover delete", | ||
| confirmation="WARNING: Deleting a storage mover will cascade delete all contained resources. This will stop all ongoing migrations and break all trust relationships with registered agents.\nAre you sure you want to delete this storage mover and all its contained resources?", | ||
| confirmation="Are you sure you want to perform this operation?", | ||
| ) | ||
| class Delete(AAZCommand): | ||
| """Deletes a Storage Mover resource. | ||
|
|
||
| :example: storage-mover delete | ||
| az storage-mover delete -g {rg} -n {mover_name} | ||
| """Delete a Storage Mover resource. | ||
| """ |
There was a problem hiding this comment.
The delete confirmation message was changed from a detailed warning (cascade delete/stop migrations/break trust) to a generic prompt. Since this operation is destructive and has important side effects, please restore a more explicit warning in the confirmation text so users understand the impact before proceeding.
| @register_command( | ||
| "storage-mover agent unregister", | ||
| confirmation="WARNING: Deleting this agent will stop ongoing migrations on this agent. Job definitions that reference this agent can’t be started until their agent reference is updated to a working agent. Registering this agent again will result in a new identity and not fix existing job definitions. Note that the Azure ARC trust is not broken. The Hybrid Compute resource must be manually removed to invalidate the agent identity that may still be allowed access to target storage containers. \nAre you sure you want to delete this storage mover agent?", | ||
| confirmation="Are you sure you want to perform this operation?", | ||
| ) | ||
| class Unregister(AAZCommand): | ||
| """Unregisters an Agent resource. | ||
|
|
||
| :example: agent unregister | ||
| az storage-mover agent unregister -g {rg} -n {agent_name} --storage-mover-name {mover_name} | ||
| """Unregisters an Agent resource | ||
| """ |
There was a problem hiding this comment.
Similar to storage-mover delete, the agent unregister confirmation was reduced to a generic prompt. Unregistering an agent can stop ongoing migrations and requires manual cleanup (e.g., Hybrid Compute resource). Please keep a more descriptive confirmation warning so users aren’t surprised by the consequences.
| @@ -44,10 +49,16 @@ interactions: | |||
| - CONFIG_NOCACHE | |||
| x-content-type-options: | |||
| - nosniff | |||
| x-ms-failure-cause: | |||
| - gateway | |||
| x-ms-operation-identifier: | |||
| - tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47,objectId=ade47690-3254-48fb-af97-0c4da9dd127c/eastus2euap/af04ffb4-5a21-4f71-ba6c-2c8802244b0c | |||
| x-ms-providerhub-traffic: | |||
There was a problem hiding this comment.
This recording now contains user identifiers in the response/headers (e.g., systemData.createdBy email and x-ms-operation-identifier with tenantId/objectId). Recordings in this repo are usually sanitized; please scrub these values (replace with placeholders) to avoid committing potentially identifying tenant/user metadata.
| def _get_args_for_endpoint_for_s3_with_hmac(endpoint_name, resource_group, storage_mover_name, | ||
| source_uri=None, source_type=None, | ||
| access_key_uri=None, secret_key_uri=None, | ||
| other_source_type_description=None, | ||
| endpoint_kind=None, description=None): | ||
| args = { | ||
| "endpoint_name": endpoint_name, "resource_group": resource_group, "storage_mover_name": storage_mover_name, | ||
| "description": description, | ||
| "endpoint_kind": endpoint_kind, | ||
| "s3_with_hmac": { | ||
| "source_uri": source_uri, | ||
| "source_type": source_type, | ||
| "other_source_type_description": other_source_type_description, | ||
| } | ||
| } | ||
| if access_key_uri is not None or secret_key_uri is not None: | ||
| args["s3_with_hmac"]["credentials"] = { | ||
| "access_key_uri": access_key_uri, | ||
| "secret_key_uri": secret_key_uri, | ||
| } | ||
| return args | ||
|
|
||
|
|
||
| def endpoint_create_for_s3_with_hmac(cmd, endpoint_name, resource_group, storage_mover_name, | ||
| source_uri, source_type, access_key_uri, secret_key_uri, | ||
| endpoint_kind=None, other_source_type_description=None, description=None): | ||
| Create_Endpoint = Create(cmd.loader) | ||
| args = _get_args_for_endpoint_for_s3_with_hmac( | ||
| endpoint_name, resource_group, storage_mover_name, | ||
| source_uri, source_type, access_key_uri, secret_key_uri, | ||
| other_source_type_description, endpoint_kind, description) | ||
| return Create_Endpoint(args) | ||
|
|
||
|
|
||
| def endpoint_update_for_s3_with_hmac(cmd, endpoint_name, resource_group, storage_mover_name, | ||
| access_key_uri=None, secret_key_uri=None, description=None): | ||
| Update_Endpoint = Update(cmd.loader) | ||
| args = _get_args_for_endpoint_for_s3_with_hmac( | ||
| endpoint_name, resource_group, storage_mover_name, | ||
| access_key_uri=access_key_uri, secret_key_uri=secret_key_uri, description=description) | ||
| return Update_Endpoint(args) |
There was a problem hiding this comment.
endpoint_update_for_s3_with_hmac calls the generated AAZ storage-mover endpoint update command and passes an s3_with_hmac object in args. However, the AAZ endpoint update schema currently only exposes azure_multi_cloud_connector and smb_mount (no s3_with_hmac), so update-for-s3-with-hmac is likely to fail argument validation at runtime. To fix, either extend the AAZ endpoint update schema to support s3_with_hmac updates, or implement this update via a dedicated AAZ command that supports the S3WithHMAC payload.
This checklist is used to make sure that common guidelines for a pull request are followed.
Related command
General Guidelines
azdev style <YOUR_EXT>locally? (pip install azdevrequired)python scripts/ci/test_index.py -qlocally? (pip install wheel==0.30.0required)For new extensions:
About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update
src/index.jsonautomatically.You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify
src/index.json.