Skip to content

Users/alsehr/849 local test #968

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AlexanderSehr merged 36 commits into from
Feb 9, 2022
Merged

Users/alsehr/849 local test #968

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
4f0acdf
First commit
AlexanderSehr Feb 4, 2022
6574b0b
Updated interface
AlexanderSehr Feb 4, 2022
9fd3f16
Refactored file logic
AlexanderSehr Feb 4, 2022
e02f58a
Further refactoring
AlexanderSehr Feb 4, 2022
b1c3955
Continued iteration
AlexanderSehr Feb 4, 2022
c6115e8
Update to latest
AlexanderSehr Feb 4, 2022
bf8d643
Futher updated structure
AlexanderSehr Feb 4, 2022
8393dbd
Updated pipelines
AlexanderSehr Feb 4, 2022
c02d00f
Added conversion
AlexanderSehr Feb 4, 2022
339ec04
Fixed dependencyies
AlexanderSehr Feb 4, 2022
8a008d4
Small fix
AlexanderSehr Feb 4, 2022
f331ffb
Updated documentation
AlexanderSehr Feb 4, 2022
22a3ce9
Cleanup
AlexanderSehr Feb 4, 2022
03cd3e6
Rollback dependency file changes
AlexanderSehr Feb 4, 2022
7ae14bb
Further rollbacks
AlexanderSehr Feb 4, 2022
9df1eaf
Small update
AlexanderSehr Feb 4, 2022
dc34b9e
Minor update
AlexanderSehr Feb 4, 2022
c09c5a0
Minor cleanup in docs
AlexanderSehr Feb 4, 2022
eb929a3
Further simplificatin
AlexanderSehr Feb 4, 2022
aecc29a
Updated dependency pipeline & docs
AlexanderSehr Feb 5, 2022
1d0f4f0
Minor fix
AlexanderSehr Feb 5, 2022
d9397c2
Updated token handling
AlexanderSehr Feb 5, 2022
3267085
Updated settings.json
AlexanderSehr Feb 8, 2022
f40a355
Minor fix
AlexanderSehr Feb 8, 2022
1792006
added missing SA conversion
AlexanderSehr Feb 8, 2022
6ab377a
Fixed further locations
AlexanderSehr Feb 8, 2022
316f07b
Renamed param
AlexanderSehr Feb 8, 2022
ca3d1cc
Small fix
AlexanderSehr Feb 8, 2022
ca35930
Changed casing
AlexanderSehr Feb 9, 2022
177fc0f
Renamed file & parameters as per comments
AlexanderSehr Feb 9, 2022
22531fb
Updated interface
AlexanderSehr Feb 9, 2022
e1cff58
Update to latest
AlexanderSehr Feb 9, 2022
b8719b3
Updated docs
AlexanderSehr Feb 9, 2022
ebee028
Changed input object buildup
AlexanderSehr Feb 9, 2022
9d21a08
Reduced verbosity
AlexanderSehr Feb 9, 2022
e0e20e7
Reduced verbosity
AlexanderSehr Feb 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 34 additions & 22 deletions .azuredevops/pipelineTemplates/jobs.validateModuleDeployment.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,35 +162,47 @@ jobs:
pwsh: true
inline: |
# Load used functions
. (Join-Path '$(moduleRepoRoot)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInParameterFile.ps1')
. (Join-Path '$(moduleRepoRoot)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFile.ps1')

# Load Settings File
$Settings = Get-Content -Path (Join-Path '$(moduleRepoRoot)' 'settings.json') | ConvertFrom-Json

# Initialize Default Parameter File Tokens
$DefaultParameterFileTokens = @(
@{ Name = 'resourceGroupName'; Value = '${{ parameters.resourceGroupName }}' }
@{ Name = 'subscriptionId'; Value = '${{ parameters.subscriptionId }}' }
@{ Name = 'managementGroupId'; Value = '${{ parameters.managementGroupId }}' }
@{ Name = "tenantId"; Value = '$(ARM_TENANT_ID)' }
@{ Name = "deploymentSpId"; Value = '$(DEPLOYMENT_SP_ID)' }
) | ForEach-Object { [PSCustomObject]$PSItem }

# Get additional Custom Parameter File Tokens from input
Write-Verbose 'Additional Custom Parameter File Tokens: ${{ deploymentBlock.customParameterFileTokens }}' -Verbose
$OtherCustomParameterFileTokens = '${{ deploymentBlock.customParameterFileTokens }}' | ConvertFrom-Json
$Settings = Get-Content -Path (Join-Path '$(moduleRepoRoot)' 'settings.json') | ConvertFrom-Json -AsHashTable

# Construct Token Function Input
$ConvertTokensInputs = @{
ParameterFilePath = Join-Path '$(parametersRepoRoot)' '${{ deploymentBlock.path }}'
DefaultParameterFileTokens = $DefaultParameterFileTokens
OtherCustomParameterFileTokens = $OtherCustomParameterFileTokens
LocalCustomParameterFileTokens = $Settings.parameterFileTokens.localTokens.tokens
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
Tokens = @{}
FilePath = Join-Path '$(parametersRepoRoot)' '${{ deploymentBlock.path }}'
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
}

# Add defaults
$ConvertTokensInputs.Tokens += @{
resourceGroupName = '${{ parameters.resourceGroupName }}'
subscriptionId = '${{ parameters.subscriptionId }}'
managementGroupId = '${{ parameters.managementGroupId }}'
tenantId = '$(ARM_TENANT_ID)'
deploymentSpId = '$(DEPLOYMENT_SP_ID)'
}

# Add local tokens
if ($Settings.parameterFileTokens.localTokens) {
$tokenMap = @{}
foreach ($token in $Settings.parameterFileTokens.localTokens) {
$tokenMap += @{ $token.name = $token.value }
}
Write-Verbose ('Using local tokens [{0}]' -f ($tokenMap.Keys -join ', ')) -Verbose
$ConvertTokensInputs.Tokens += $tokenMap
}

# Add custom tokens (passed in via the pipeline)
if(-not [String]::IsNullOrEmpty('${{ deploymentBlock.customParameterFileTokens }}')) {
$customTokens = '${{ deploymentBlock.customParameterFileTokens }}' | ConvertFrom-Json -AsHashTable
Write-Verbose ('Using custom parameter file tokens [{0}]' -f ($customTokens.Keys -join ', ')) -Verbose
$ConvertTokensInputs.Tokens += $customTokens
}

# Invoke Token Replacement Functionality
$null = Convert-TokensInParameterFile @ConvertTokensInputs -Verbose
$null = Convert-TokensInFile @ConvertTokensInputs

# [Validation] task(s)
#---------------------
Expand Down
99 changes: 64 additions & 35 deletions .azuredevops/platformPipelines/platform.dependencies.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,21 +186,30 @@ stages:
azureSubscription: $(serviceConnection)
ScriptType: 'InlineScript'
Inline: |

$parameterFilePath = Join-Path '$(Build.SourcesDirectory)' '$(dependencyPath)' '$(resourceType)' 'parameters' 'parameters.json'
# Load used functions
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'sharedScripts' 'Export-ContentToBlob.ps1')
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInParameterFile.ps1')
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFile.ps1')

# Replace tokens in parameter file
$Settings = Get-Content -Path (Join-Path '$(Build.SourcesDirectory)' 'settings.json') | ConvertFrom-Json
$Settings = Get-Content -Path (Join-Path '$(Build.SourcesDirectory)' 'settings.json') | ConvertFrom-Json -AsHashTable
$ConvertTokensInputs = @{
ParameterFilePath = $parameterFilePath
LocalCustomParameterFileTokens = $Settings.parameterFileTokens.localTokens.tokens
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
FilePath = $parameterFilePath
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
}

# Add local tokens
if ($Settings.parameterFileTokens.localTokens) {
$tokenMap = @{}
foreach ($token in $Settings.parameterFileTokens.localTokens) {
$tokenMap += @{ $token.name = $token.value }
}
Write-Verbose ('Using local tokens [{0}]' -f ($tokenMap.Keys -join ', ')) -Verbose
$ConvertTokensInputs.Tokens = $tokenMap
}
$null = Convert-TokensInParameterFile @ConvertTokensInputs -Verbose

$null = Convert-TokensInFile @ConvertTokensInputs

# Get storage account name
$storageAccountParameters = (ConvertFrom-Json (Get-Content -path $parameterFilePath -Raw)).parameters
Expand Down Expand Up @@ -307,14 +316,21 @@ stages:
Inline: |

# Load used functions
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInParameterFile.ps1')
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFile.ps1')

# Prepare replace tokens in parameter file
$Settings = Get-Content -Path (Join-Path '$(Build.SourcesDirectory)' 'settings.json') | ConvertFrom-Json
$Settings = Get-Content -Path (Join-Path '$(Build.SourcesDirectory)' 'settings.json') | ConvertFrom-Json -AsHashTable
$ConvertTokensInputs = @{
LocalCustomParameterFileTokens = $Settings.parameterFileTokens.localTokens.tokens
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
}
if ($Settings.parameterFileTokens.localTokens) {
$tokenMap = @{}
foreach ($token in $Settings.parameterFileTokens.localTokens) {
$tokenMap += @{ $token.name = $token.value }
}
Write-Verbose ('Using local tokens [{0}]' -f ($tokenMap.Keys -join ', ')) -Verbose
$ConvertTokensInputs.Tokens = $tokenMap
}

# Retrieving parameters from previous job outputs and parameter files
Expand All @@ -324,12 +340,12 @@ stages:

Write-Verbose "Retrieving parameters from storage account parameter files" -Verbose
$parameterFilePath = Join-Path '$(Build.SourcesDirectory)' '$(dependencyPath)' '$(saResourceType)' 'parameters' 'parameters.json'
$null = Convert-TokensInParameterFile @ConvertTokensInputs -ParameterFilePath $parameterFilePath -Verbose
$null = Convert-TokensInFile @ConvertTokensInputs -FilePath $parameterFilePath -Verbose
$storageAccountParameters = (ConvertFrom-Json (Get-Content -path $parameterFilePath -Raw)).parameters

Write-Verbose "Retrieving parameters from image template parameter files" -Verbose
$parameterFilePath = Join-Path '$(Build.SourcesDirectory)' '$(dependencyPath)' '$(resourceType)' 'parameters' 'parameters.json'
$null = Convert-TokensInParameterFile @ConvertTokensInputs -ParameterFilePath $parameterFilePath -Verbose
$null = Convert-TokensInFile @ConvertTokensInputs -FilePath $parameterFilePath -Verbose
$imageTemplateParameters = (ConvertFrom-Json (Get-Content -path $parameterFilePath -Raw)).parameters

# Initializing parameters before the blob copy
Expand Down Expand Up @@ -540,7 +556,7 @@ stages:
- path: $(dependencyPath)/$(resourceType)/parameters/parameters.json
templateFilePath: $(templateFilePath)
displayName: Default recovery services vault
customParameterFileTokens: '[{"Name":"msiPrincipalId","Value":"$(msiPrincipalId)"}]'
customParameterFileTokens: '{"msiPrincipalId":"$(msiPrincipalId)"}'

- stage: deploy_kv
displayName: Deploy key vaults
Expand All @@ -561,17 +577,17 @@ stages:
templateFilePath: $(templateFilePath)
displayName: Default Key Vault
jobName: default_kv
customParameterFileTokens: '[{"Name":"msiPrincipalId","Value":"$(msiPrincipalId)"}]'
customParameterFileTokens: '{"msiPrincipalId":"$(msiPrincipalId)"}'
- path: $(dependencyPath)/$(resourceType)/parameters/pe.parameters.json
templateFilePath: $(templateFilePath)
displayName: Private Endpoint Key Vault
customParameterFileTokens: '[{"Name":"msiPrincipalId","Value":"$(msiPrincipalId)"}]'
customParameterFileTokens: '{"msiPrincipalId":"$(msiPrincipalId)"}'
- ${{ if eq( parameters.deploySqlMiDependencies, true) }}:
- path: $(dependencyPath)/$(resourceType)/parameters/sqlmi.parameters.json
templateFilePath: $(templateFilePath)
displayName: SQLMI key vault
jobName: sqlmi_kv
customParameterFileTokens: '[{"Name":"msiPrincipalId","Value":"$(msiPrincipalId)"}]'
customParameterFileTokens: '{"msiPrincipalId":"$(msiPrincipalId)"}'
- job:
displayName: Set key vault secrets keys and certificates
dependsOn:
Expand Down Expand Up @@ -604,20 +620,26 @@ stages:
azureSubscription: $(serviceConnection)
ScriptType: 'InlineScript'
Inline: |

$parameterFilePath = Join-Path '$(Build.SourcesDirectory)' '$(dependencyPath)' '$(resourceType)' 'parameters' 'parameters.json'
# Load used functions
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInParameterFile.ps1')
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFile.ps1')

# Replace tokens in parameter file
$Settings = Get-Content -Path (Join-Path '$(Build.SourcesDirectory)' 'settings.json') | ConvertFrom-Json
$Settings = Get-Content -Path (Join-Path '$(Build.SourcesDirectory)' 'settings.json') | ConvertFrom-Json -AsHashTable
$ConvertTokensInputs = @{
ParameterFilePath = $parameterFilePath
LocalCustomParameterFileTokens = $Settings.parameterFileTokens.localTokens.tokens
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
FilePath = $parameterFilePath
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
}
$null = Convert-TokensInParameterFile @ConvertTokensInputs -Verbose
if ($Settings.parameterFileTokens.localTokens) {
$tokenMap = @{}
foreach ($token in $Settings.parameterFileTokens.localTokens) {
$tokenMap += @{ $token.name = $token.value }
}
Write-Verbose ('Using local tokens [{0}]' -f ($tokenMap.Keys -join ', ')) -Verbose
$ConvertTokensInputs.Tokens = $tokenMap
}
$null = Convert-TokensInFile @ConvertTokensInputs

# Get key vault name
$keyVaultParameters = (ConvertFrom-Json (Get-Content -Path $parameterFilePath -Raw)).parameters
Expand Down Expand Up @@ -704,17 +726,24 @@ stages:

$parameterFilePath = Join-Path '$(Build.SourcesDirectory)' '$(dependencyPath)' '$(resourceType)' 'parameters' 'sqlmi.parameters.json'
# Load used functions
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInParameterFile.ps1')
. (Join-Path '$(Build.SourcesDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFile.ps1')

# Replace tokens in parameter file
$Settings = Get-Content -Path (Join-Path '$(Build.SourcesDirectory)' 'settings.json') | ConvertFrom-Json
$Settings = Get-Content -Path (Join-Path '$(Build.SourcesDirectory)' 'settings.json') | ConvertFrom-Json -AsHashTable
$ConvertTokensInputs = @{
ParameterFilePath = $parameterFilePath
LocalCustomParameterFileTokens = $Settings.parameterFileTokens.localTokens.tokens
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
FilePath = $parameterFilePath
TokenPrefix = $Settings.parameterFileTokens.tokenPrefix
TokenSuffix = $Settings.parameterFileTokens.tokenSuffix
}
if ($Settings.parameterFileTokens.localTokens) {
$tokenMap = @{}
foreach ($token in $Settings.parameterFileTokens.localTokens) {
$tokenMap += @{ $token.name = $token.value }
}
Write-Verbose ('Using local tokens [{0}]' -f ($tokenMap.Keys -join ', ')) -Verbose
$ConvertTokensInputs.Tokens = $tokenMap
}
$null = Convert-TokensInParameterFile @ConvertTokensInputs -Verbose
$null = Convert-TokensInFile @ConvertTokensInputs

# Get key vault name
$keyVaultParameters = (ConvertFrom-Json (Get-Content -Path $parameterFilePath -Raw)).parameters
Expand Down Expand Up @@ -777,7 +806,7 @@ stages:
- path: $(dependencyPath)/$(resourceType)/parameters/parameters.json
templateFilePath: $(templateFilePath)
displayName: MSI Role Assignment
customParameterFileTokens: '[{"Name":"msiPrincipalId","Value":"$(msiPrincipalId)"}]'
customParameterFileTokens: '{"msiPrincipalId":"$(msiPrincipalId)"}'

- stage: deploy_vnet
displayName: Deploy virtual networks
Expand Down
Loading