Skip to content

Fix VLW container cleanup in Resource-Helpers to handle soft-deleted blob versions#14973

Open
ibrandes wants to merge 2 commits intomainfrom
storage/supportSoftDeletedBlobsDuringResourceCleanup
Open

Fix VLW container cleanup in Resource-Helpers to handle soft-deleted blob versions#14973
ibrandes wants to merge 2 commits intomainfrom
storage/supportSoftDeletedBlobsDuringResourceCleanup

Conversation

@ibrandes
Copy link
Copy Markdown
Member

@ibrandes ibrandes commented Apr 7, 2026

Summary

  • Added Remove-VlwContainerBlobs function that lists blobs with -IncludeVersion -IncludeDeleted and clears immutability policies / legal holds before deleting each version across multiple passes
  • Routed VLW containers through the new function in RemoveStorageAccount, leaving non-VLW containers on the existing EnableBlobDeletion path

Problem

When soft-delete is enabled on a storage account, deleting a blob in a VLW container creates a non-current version instead of truly removing it. The existing cleanup in RemoveStorageAccount uses a basic Get-AzStorageBlob listing that can't see these leftover versions. They silently block container deletion, causing the management plane DELETE to return 409 Conflict.

Approach

The new Remove-VlwContainerBlobs function:

  1. Lists all blob versions and soft-deleted blobs (-IncludeVersion -IncludeDeleted)
  2. Unconditionally clears legal holds and immutability policies on each item (swallowing errors for blobs that don't have them)
  3. Deletes each item, falling back to base-blob deletion when deleting the current version by version ID returns 403 (OperationNotAllowedOnRootBlob)
  4. Repeats for up to 5 passes, since deleting a current version can surface new non-current versions that need their own cleanup

@ibrandes ibrandes changed the title Adding support for soft deleted blobs and blob versions in resource cleanup helper script Fix VLW container cleanup in Resource-Helpers to handle soft-deleted blob versions Apr 7, 2026
@azure-sdk
Copy link
Copy Markdown
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves storage account cleanup in Resource-Helpers.ps1 by adding VLW (versioned immutable) container-specific blob deletion that can see and delete soft-deleted blobs and non-current versions, preventing container deletion failures when soft-delete is enabled.

Changes:

  • Route VLW containers through a new version-aware cleanup path in RemoveStorageAccount.
  • Add Remove-VlwContainerBlobs to enumerate -IncludeVersion -IncludeDeleted, clear legal hold/immutability, and delete blobs over multiple passes.

Comment on lines +544 to +545
$blobs = @($Container | Get-AzStorageBlob -IncludeVersion -IncludeDeleted -ErrorAction SilentlyContinue)

Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Get-AzStorageBlob is invoked with -ErrorAction SilentlyContinue. If listing fails (auth/network/unsupported flags), this function will silently do nothing and container deletion will still fail later with minimal diagnostics. Consider using -ErrorAction Stop with a try/catch that emits a warning/error (or at least logs when listing fails) so failures are actionable.

Suggested change
$blobs = @($Container | Get-AzStorageBlob -IncludeVersion -IncludeDeleted -ErrorAction SilentlyContinue)
try {
$blobs = @($Container | Get-AzStorageBlob -IncludeVersion -IncludeDeleted -ErrorAction Stop)
} catch {
Write-Warning "Failed to list blobs for VLW container '$($Container.Name)' in storage account '$StorageAccountName'. Cleanup cannot continue for this container. $($_.Exception.Message)"
break
}

Copilot uses AI. Check for mistakes.
Comment on lines +553 to +560
try {
$blob | Remove-AzStorageBlob -Force
} catch {
# Deleting the current version by version ID returns 403
# (OperationNotAllowedOnRootBlob); fall back to base blob deletion.
try {
Remove-AzStorageBlob -Container $Container.Name -Blob $blob.Name -Context $Container.Context -Force
} catch { }
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The catch around versioned deletion falls back to deleting the base blob for any exception, not just the expected 403 OperationNotAllowedOnRootBlob scenario. This can mask real failures (e.g., permission/timeout) and still attempt a different destructive operation. Consider narrowing the fallback to the specific error code/message you expect and logging/rethrowing unexpected errors.

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@ibrandes ibrandes marked this pull request as ready for review April 8, 2026 18:25
@ibrandes ibrandes requested a review from a team as a code owner April 8, 2026 18:25
@azure-sdk
Copy link
Copy Markdown
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants