[Utilities] Added ManagementGroup-Deployment removal script

docs/wiki/The CI environment - Deployment validation.md

@@ -35,7 +35,9 @@ If any of these parallel deployments require multiple/different/specific resourc
 
 The parameter files used in this stage should ideally cover as many configurations as possible to validate the template flexibility, i.e., to verify that the module can cover multiple scenarios in which the given Azure resource may be used. Using the example of the CosmosDB module, we may want to have one parameter file for the minimum amount of required parameters, one parameter file for each CosmosDB type to test individual configurations, and at least one parameter file testing the supported extension resources such as RBAC & diagnostic settings.
 
-> **Note**: Since every customer environment might be different due to applied Azure Policies or security policies, modules might behave differently and naming conventions need to be verified beforehand.
+> **Note:** Since every customer environment might be different due to applied Azure Policies or security policies, modules might behave differently and naming conventions need to be verified beforehand.
+
+> **Note:** Management-Group deployments may eventually exceed the limit of [800](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#management-group-limits) and require you to remove some of them manually. If you are faced with any corresponding error message you can manually remove deployments on a Management-Group-Level on scale using one of our [utilities](./The%20CI%20environment%20-%20Management%20Group%20Deployment%20removal%20utility).
 
 ### Output example
 

docs/wiki/The CI environment - Management Group Deployment removal utility.md

@@ -0,0 +1,26 @@
+Use this script to remove Management-Group-Level Azure deployments on scale. This may be necessary in cases where you run many (test) deployments in this scope as Azure currently only auto-removes deployments from an [Resource-Group & Subscription](https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-history-deletions?tabs=azure-powershell) scope. The resulting error message may look similar to `Creating the deployment '<deploymentName>' would exceed the quota of '800'. The current deployment count is '804'. Please delete some deployments before creating a new one, or see https://aka.ms/800LimitFix for information on managing deployment limits.`
+
+---
+
+### _Navigation_
+
+- [Location](#location)
+- [How it works](#how-it-works)
+- [How to use it](#how-to-use-it)
+
+---
+# Location
+
+You can find the script under [`/utilities/tools/Clear-ManagementGroupDeployment`](https://github.com/Azure/ResourceModules/blob/main/utilities/tools/Clear-ManagementGroupDeployment.ps1)
+
+# How it works
+
+1. The script fetches all current deployments from Azure.
+1. By default it then filters them down to non-running & non-failing deployments (can be modified).
+1. Lastly, it removes all matching deployments in chunks of 100 deployments each.
+
+# How to use it
+
+For details on how to use the function, please refer to the script's local documentation.
+
+> **Note:** The script must be loaded ('*dot-sourced*') before the function can be invoked.

utilities/tools/Clear-ManagementGroupDeployment.ps1

@@ -0,0 +1,101 @@
+
+<#
+.SYNOPSIS
+Bulk delete all deployments on the given management group scope
+
+.DESCRIPTION
+Bulk delete all deployments on the given management group scope
+
+.PARAMETER ManagementGroupId
+Mandatory. The Resource ID of the Management Group to remove the deployments for.
+
+.PARAMETER DeploymentStatusToExclude
+Optional. The status to exlude from removals. Can be multiple. By default, we exclude any deployment that is in state 'running' or 'failed'.
+
+.EXAMPLE
+Clear-ManagementGroupDeployment -ManagementGroupId 'MyManagementGroupId'
+
+Bulk remove all 'non-running' & 'non-failed' deployments from the Management Group with ID 'MyManagementGroupId'
+
+.EXAMPLE
+Clear-ManagementGroupDeployment -ManagementGroupId 'MyManagementGroupId' -DeploymentStatusToExclude @('running')
+
+Bulk remove all 'non-running' deployments from the Management Group with ID 'MyManagementGroupId'
+#>
+function Clear-ManagementGroupDeployment {
+
+
+    [CmdletBinding(SupportsShouldProcess)]
+    param (
+        [Parameter(Mandatory = $true)]
+        [string] $ManagementGroupId,
+
+        [Parameter(Mandatory = $false)]
+        [string[]] $DeploymentStatusToExclude = @('running', 'failed')
+    )
+
+    # Load used functions
+    . (Join-Path $PSScriptRoot 'helper' 'Split-Array.ps1')
+
+    $getInputObject = @{
+        Method  = 'GET'
+        Uri     = "https://management.azure.com/providers/Microsoft.Management/managementGroups/$ManagementGroupId/providers/Microsoft.Resources/deployments/?api-version=2021-04-01"
+        Headers = @{
+            Authorization = 'Bearer {0}' -f (Get-AzAccessToken).Token
+        }
+    }
+    $response = Invoke-RestMethod @getInputObject
+
+    if (($response | Get-Member -MemberType 'NoteProperty').Name -notcontains 'value') {
+        throw ('Fetching deployments failed with error [{0}]' -f ($reponse | Out-String))
+    }
+
+    $relevantDeployments = $response.value | Where-Object { $_.properties.provisioningState -notin $DeploymentStatusToExclude }
+
+    if (-not $relevantDeployments) {
+        Write-Verbose 'No deployments found' -Verbose
+        return
+    }
+
+    $relevantDeploymentChunks = , (Split-Array -InputArray $relevantDeployments -SplitSize 100)
+
+    Write-Verbose ('Triggering the removal of [{0}] deployments of management group [{1}]' -f $relevantDeployments.Count, $ManagementGroupId)
+
+    $failedRemovals = 0
+    $successfulRemovals = 0
+    foreach ($deployments in $relevantDeploymentChunks) {
+
+        $requests = $deployments | ForEach-Object {
+            @{ httpMethod            = 'DELETE'
+                name                 = (New-Guid).Guid
+                requestHeaderDetails = @{
+                    commandName = 'HubsExtension.Microsoft.Resources/deployments.BulkDelete.execute'
+                }
+                url                  = '/providers/Microsoft.Management/managementGroups/{0}/providers/Microsoft.Resources/deployments/{1}?api-version=2019-08-01' -f $ManagementGroupId, $_.name
+            }
+        }
+
+        $removeInputObject = @{
+            Method  = 'POST'
+            Uri     = 'https://management.azure.com/batch?api-version=2020-06-01'
+            Headers = @{
+                Authorization  = 'Bearer {0}' -f (Get-AzAccessToken).Token
+                'Content-Type' = 'application/json'
+            }
+            Body    = @{
+                requests = $requests
+            } | ConvertTo-Json -Depth 4
+        }
+        if ($PSCmdlet.ShouldProcess(('Removal of [{0}] deployments' -f $requests.Count), 'Request')) {
+            $response = Invoke-RestMethod @removeInputObject
+
+            $failedRemovals += ($response.responses | Where-Object { $_.httpStatusCode -notlike '20*' }  ).Count
+            $successfulRemovals += ($response.responses | Where-Object { $_.httpStatusCode -like '20*' }  ).Count
+        }
+    }
+
+    Write-Verbose 'Outcome' -Verbose
+    Write-Verbose '=======' -Verbose
+    Write-Verbose "Successful removals:`t`t$successfulRemovals" -Verbose
+    Write-Verbose "Un-successful removals:`t$failedRemovals" -Verbose
+}

utilities/tools/helper/Split-Array.ps1

@@ -0,0 +1,47 @@
+<#
+.SYNOPSIS
+Split a given array evenly into chunks of n-items
+
+.DESCRIPTION
+Split a given array evenly into chunks of n-item
+
+.PARAMETER InputArray
+Mandatory. The array to split
+
+.PARAMETER SplitSize
+Mandatory. The chunk size to split into.
+
+.EXAMPLE
+Split-Array -InputArray @('1','2,'3','4','5') -SplitSize 3
+
+Split the given array @('1','2,'3','4','5') into chunks of size '3'. Will return the multi-demensional array @(@('1','2,'3'),@('4','5'))
+#>
+function Split-Array {
+
+    [CmdletBinding()]
+    param(
+        [Parameter(Mandatory = $true)]
+        [object[]] $InputArray,
+
+        [Parameter(Mandatory = $true)]
+        [int] $SplitSize
+    )
+    begin {
+        Write-Debug ('{0} entered' -f $MyInvocation.MyCommand)
+    }
+    process {
+
+        if ($splitSize -ge $InputArray.Count) {
+            return $InputArray
+        } else {
+            $res = @()
+            for ($Index = 0; $Index -lt $InputArray.Count; $Index += $SplitSize) {
+                $res += , ( $InputArray[$index..($index + $splitSize - 1)] )
+            }
+            return $res
+        }
+    }
+    end {
+        Write-Debug ('{0} existed' -f $MyInvocation.MyCommand)
+    }
+}