Skip to content

Adjust release pipeline to use cdpx acr #647

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sarahpeiffer merged 6 commits into from
Sep 21, 2021
Merged

Adjust release pipeline to use cdpx acr #647

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b30d369
Adjust release pipeline to use cdpx acr
sarahpeiffer Sep 20, 2021
dd026b8
Merge branch 'ci_dev' into sarah/pipeline
sarahpeiffer Sep 20, 2021
eb9fe04
Adjust release pipeline to use cdpx acr
sarahpeiffer Sep 20, 2021
d90fc21
Merge branch 'sarah/pipeline' of https://github.com/microsoft/Docker-…
sarahpeiffer Sep 20, 2021
6d1fe13
Update CDPX ACR path
sarahpeiffer Sep 20, 2021
eae0f53
Add check for cdpx repo variable
sarahpeiffer Sep 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .pipelines/pipeline.user.linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,7 @@ package:
repository_name: 'cdpxlinux' # only supported ones are cdpx acr repos
tag: 'cidev' # OPTIONAL: Defaults to latest. The tag for the built image. Final tag will be 1.0.0alpha, 1.0.0-timestamp-commitID.
latest: false # OPTIONAL: Defaults to false. If tag is not set to latest and this flag is set, then tag as latest as well and push latest as well.
publish_unique_tag: true # If set, the image in the registry is tagged with the unique tag generated by CDPx
metadata_file:
artifact_path: 'linux-image-meta.json' # If defined, the drop outputs relative path to the file into which JSON metadata about the created image is emitted.
export_to_artifact_path: 'agentimage.tar.gz' # path for exported image and use this instead of fixed tag
3 changes: 3 additions & 0 deletions .pipelines/pipeline.user.windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,7 @@ package:
repository_name: 'cdpxwin1809' # only supported ones are cdpx acr repos
tag: 'win-cidev' # OPTIONAL: Defaults to latest. The tag for the built image. Final tag will be 1.0.0alpha, 1.0.0-timestamp-commitID.
latest: false # OPTIONAL: Defaults to false. If tag is not set to latest and this flag is set, then tag as latest as well and push latest as well.
publish_unique_tag: true # If set, the image in the registry is tagged with the unique tag generated by CDPx
metadata_file:
artifact_path: 'windows-image-meta.json' # If defined, the drop outputs relative path to the file into which JSON metadata about the created image is emitted.
export_to_artifact_path: 'agentimage.tar.zip' # path for exported image and use this instead of fixed tag
40 changes: 16 additions & 24 deletions ...ment/agent-deployment/ServiceGroupRoot/Parameters/ContainerInsights.Linux.Parameters.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,30 +27,6 @@
"pushAgentToAcr.sh"
],
"environmentVariables": [
{
"name": "AGENT_IMAGE_URI",
"value": "__CONTAINER_URI__"
},
{
"name": "AGENT_IMAGE_SAS",
"value": "__CONTAINER_SAS_TOKEN__"
},
{
"name": "STORAGE_CONTAINER_NAME",
"value": "__STORAGE_CONTAINER_NAME__"
},
{
"name": "STORAGE_ACCOUNT_NAME",
"value": "__STORAGE_ACCOUNT_NAME__"
},
{
"name": "AGENT_IMAGE_TAR_FILE_NAME",
"value": "agentimage.tar.gz"
},
{
"name": "RELEASE_ID",
"value": "__RELEASE_ID__"
},
{
"name": "ACR_NAME",
"value": "__ACR_NAME__"
Expand All @@ -66,6 +42,22 @@
{
"name": "AGENT_IMAGE_FULL_PATH",
"value": "__ACR_NAME__/public/azuremonitor/containerinsights/__AGENT_RELEASE__:__AGENT_RELEASE____AGENT_IMAGE_TAG_SUFFIX__"
},
{
"name": "CDPX_ACR",
"value": "__CDPX_LINUX_ACR__"
},
{
"name": "CDPX_REGISTRY",
"value": "__CDPX_LINUX_REGISTRY__"
},
{
"name": "CDPX_REPO_NAME",
"value": "__CDPX_LINUX_REPO_NAME__"
},
{
"name": "CDPX_TAG",
"value": "__CDPX_LINUX_TAG__"
}
],
"identity": {
Expand Down
44 changes: 16 additions & 28 deletions ...nt/agent-deployment/ServiceGroupRoot/Parameters/ContainerInsights.Windows.Parameters.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,34 +27,6 @@
"pushAgentToAcr.sh"
],
"environmentVariables": [
{
"name": "WINDOWS",
"value": "win-"
},
{
"name": "AGENT_IMAGE_URI",
"value": "__CONTAINER_URI__"
},
{
"name": "AGENT_IMAGE_SAS",
"value": "__CONTAINER_SAS_TOKEN__"
},
{
"name": "STORAGE_CONTAINER_NAME",
"value": "__STORAGE_CONTAINER_NAME__"
},
{
"name": "STORAGE_ACCOUNT_NAME",
"value": "__STORAGE_ACCOUNT_NAME__"
},
{
"name": "AGENT_IMAGE_TAR_FILE_NAME",
"value": "agentimage.tar.zip"
},
{
"name": "RELEASE_ID",
"value": "__RELEASE_ID__"
},
{
"name": "ACR_NAME",
"value": "__ACR_NAME__"
Expand All @@ -70,6 +42,22 @@
{
"name": "AGENT_IMAGE_FULL_PATH",
"value": "__ACR_NAME__/public/azuremonitor/containerinsights/__AGENT_RELEASE__:win-__AGENT_RELEASE____AGENT_IMAGE_TAG_SUFFIX__"
},
{
"name": "CDPX_ACR",
"value": "__CDPX_WINDOWS_ACR__"
},
{
"name": "CDPX_REGISTRY",
"value": "__CDPX_WINDOWS_REGISTRY__"
},
{
"name": "CDPX_REPO_NAME",
"value": "__CDPX_WINDOWS_REPO_NAME__"
},
{
"name": "CDPX_TAG",
"value": "__CDPX_WINDOWS_TAG__"
}
],
"identity": {
Expand Down
41 changes: 26 additions & 15 deletions deployment/agent-deployment/ServiceGroupRoot/ScopeBindings/Public.ScopeBindings.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,31 +17,42 @@
"find": "__AGENT_IMAGE_TAG_SUFFIX__",
"replaceWith": "$(AgentImageTagSuffix)"
},
{
"find": "__RELEASE_ID__",
"replaceWith": "$(Release.ReleaseId)"
},
{
"find": "__MANAGED_IDENTITY__",
"replaceWith": "$(ManagedIdentity)"
},
{
"find": "__CONTAINER_URI__",
"replaceWith": "$(Storage.StorageContainerUri)"
},
"find": "__CDPX_LINUX_ACR__",
"replaceWith": "$(CDPXLinuxACR)"
},
{
"find": "__CONTAINER_SAS_TOKEN__",
"replaceWith": "$(Storage.StorageContainerSasToken)"
},
"find": "__CDPX_WINDOWS_ACR__",
"replaceWith": "$(CDPXWindowsACR)"
},
{
"find": "__CDPX_LINUX_REGISTRY__",
"replaceWith": "$(CDPXLinuxRegistry)"
},
{
"find": "__CDPX_WINDOWS_REGISTRY__",
"replaceWith": "$(CDPXWindowsRegistry)"
},
{
"find": "__CDPX_LINUX_TAG__",
"replaceWith": "$(CDPXLinuxTag)"
},
{
"find": "__CDPX_WINDOWS_TAG__",
"replaceWith": "$(CDPXWindowsTag)"
},
{
"find": "__STORAGE_CONTAINER_NAME__",
"replaceWith": "$(StorageContainerName)"
"find": "__CDPX_LINUX_REPO_NAME__",
"replaceWith": "$(CDPXLinuxRepoName)"
},
{
"find": "__STORAGE_ACCOUNT_NAME__",
"replaceWith": "$(StorageAccountName)"
"find": "__CDPX_WINDOWS_REPO_NAME__",
"replaceWith": "$(CDPXWindowsRepoName)"
}

]
}
]
Expand Down
102 changes: 25 additions & 77 deletions deployment/agent-deployment/ServiceGroupRoot/Scripts/pushAgentToAcr.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,24 +12,29 @@ if [ -z $AGENT_RELEASE ]; then
echo "-e error AGENT_RELEASE shouldnt be empty. check release variables"
exit 1
fi
#!
if [ -z $AGENT_IMAGE_FULL_PATH ]; then
echo "-e error AGENT_IMAGE_FULL_PATH shouldnt be empty. check release variables"
exit 1
fi

if [ -z $AGENT_IMAGE_URI ]; then
echo "-e error value of AGENT_IMAGE_URI shouldn't be empty. check output from file copy release task"
if [ -z $CDPX_ACR ]; then
echo "-e error value of CDPX_ACR shouldn't be empty. check release variables"
exit 1
fi

if [ -z $AGENT_IMAGE_SAS ]; then
echo "-e error value of AGENT_IMAGE_SAS shouldn't be empty. check output from file copy release task"
if [ -z $CDPX_TAG ]; then
echo "-e error value of CDPX_TAG shouldn't be empty. check release variables"
exit 1
fi

if [ -z $STORAGE_CONTAINER_NAME ]; then
echo "-e error value of STORAGE_CONTAINER_NAME shouldn't be empty. check release variables"
if [ -z $CDPX_REGISTRY ]; then
echo "-e error value of CDPX_REGISTRY shouldn't be empty. check release variables"
exit 1
fi

if [ -z $STORAGE_ACCOUNT_NAME ]; then
echo "-e error value of STORAGE_ACCOUNT_NAME shouldn't be empty. check release variables"
if [ -z $CDPX_REPO_NAME ]; then
echo "-e error value of CDPX_REPO_NAME shouldn't be empty. check release variables"
exit 1
fi

Expand All @@ -38,79 +43,22 @@ if [ -z $ACR_NAME ]; then
exit 1
fi

#Download agentimage tarball from blob storage account
echo "Downloading tarball image from $WINDOWS $AGENT_IMAGE_URI"
wget -O $AGENT_IMAGE_TAR_FILE_NAME "${AGENT_IMAGE_URI}${WINDOWS}${RELEASE_ID}${AGENT_IMAGE_SAS}"


if [ ! -f $AGENT_IMAGE_TAR_FILE_NAME ]; then
echo "Agent tarfile: ${AGENT_IMAGE_TAR_FILE_NAME} does not exist, unable to continue"
exit 1
fi

#Install crane
echo "Installing crane"
wget -O crane.tar.gz https://github.com/google/go-containerregistry/releases/download/v0.4.0/go-containerregistry_Linux_x86_64.tar.gz
if [ $? -eq 0 ]; then
echo "crane downloaded successfully"
else
echo "-e error crane download failed"
exit 1
fi
tar xzvf crane.tar.gz
echo "Installed crane"


#Login to az cli and authenticate to acr
echo "Login cli using managed identity"
az login --identity

echo "Getting acr credentials"
TOKEN_QUERY_RES=$(az acr login -n "$ACR_NAME" -t)
TOKEN=$(echo "$TOKEN_QUERY_RES" | jq -r '.accessToken')
if [ -z $TOKEN ]; then
echo "-e error failed to get az acr login token"
if [ $? -eq 0 ]; then
echo "Logged in successfully"
else
echo "-e error failed to login to az with managed identity credentials"
exit 1
fi

DESTINATION_ACR=$(echo "$TOKEN_QUERY_RES" | jq -r '.loginServer')
if [ -z $DESTINATION_ACR ]; then
echo "-e error value of DESTINATION_ACR shouldnt be empty"
fi

echo "Pushing ${AGENT_IMAGE_FULL_PATH} to ${ACR_NAME}"
az acr import --name $ACR_NAME --registry $CDPX_REGISTRY --source ${CDPX_ACR}/official/${CDPX_REPO_NAME}:${CDPX_TAG} --image $AGENT_IMAGE_FULL_PATH
if [ $? -eq 0 ]; then
echo "Retagged and pushed image successfully"
else
echo "-e error failed to retag and push image to destination ACR"
exit 1
fi

./crane auth login "$DESTINATION_ACR" -u "00000000-0000-0000-0000-000000000000" -p "$TOKEN"

#Prepare tarball and push to acr
if [[ "$AGENT_IMAGE_TAR_FILE_NAME" == *"tar.gz"* ]]; then
gunzip $AGENT_IMAGE_TAR_FILE_NAME
fi

if [[ "$AGENT_IMAGE_TAR_FILE_NAME" == *"tar.zip"* ]]; then
apt-get -y install unzip
if [ $? -eq 0 ]; then
unzip $AGENT_IMAGE_TAR_FILE_NAME
else
echo "-e error failed to install unzip package and cannot unzip windows agent tarball"
exit 1
fi
fi

echo "Pushing file $TARBALL_IMAGE_FILE to $AGENT_IMAGE_FULL_PATH"
./crane push *.tar "$AGENT_IMAGE_FULL_PATH"


#Delete agentimage tarball from blob storage to prevent future conflicts
echo "Deleting agentimage copy from blob storage"

BLOB_EXIST_RESULT=$(az storage blob exists --container-name $STORAGE_CONTAINER_NAME --name $WINDOWS$RELEASE_ID --account-name $STORAGE_ACCOUNT_NAME --sas-token $AGENT_IMAGE_SAS)
BLOB_EXIST=$(echo "$BLOB_EXIST_RESULT" | jq -r '.exists')
echo $BLOB_EXIST_RESULT
echo $BLOB_EXIST

if $BLOB_EXIST; then
az storage blob delete --container-name "${STORAGE_CONTAINER_NAME}" --name "${WINDOWS}${RELEASE_ID}" --account-name "${STORAGE_ACCOUNT_NAME}" --sas-token "${AGENT_IMAGE_SAS}"
echo "Deleted agentimate copy from blob storage"
else
echo "Agentimage has already been deleted from blob storage"
fi