Adopt wider tagging model #41
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,162 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| # the first argument must be the name of the container, i.e. st2, st2actionrunner, st2stream | ||
| component=$1 | ||
| # the 2nd argument is the version of the current build and expects at least major.miinor to be provided | ||
| # i.e. 3.3.0, 3.3, 2.10.5 | ||
| build_version=$2 | ||
|
|
||
| showHelp() { | ||
| echo "Use this script to determine the right tags to be attached to your new image." | ||
| echo | ||
| echo "Syntax: $0 st2component image_version" | ||
| echo "st2component = an valid image of an existing stackstorm component (i.e. st2, st2api, st2stream)" | ||
| echo "image_version = the st2 release used for your current build (i.e. 3.3.0)" | ||
| echo | ||
| echo "Possible output:" | ||
| echo "0 = no additional tags to be updated" | ||
| echo "1 = update the major.minor tag (i.e. 3.3)" | ||
| echo "2 = update the major and the major.minor tag (i.e. 3 and 3.3)" | ||
| echo "3 = update the major and the major.minor tag (i.e. 3 and 3.3) as well as the latest tag" | ||
| echo | ||
| echo "Run $0 -h|--help to show this usage information." | ||
| } | ||
|
|
||
| case $1 in | ||
| -h|--help) | ||
| showHelp | ||
| exit | ||
| ;; | ||
| esac | ||
|
|
||
| if [ "$#" -ne 2 ]; then | ||
| echo "Error: Missing or unexpected number of positional arguments. Expected: 2" | ||
| showHelp | ||
| exit 1 | ||
| fi | ||
|
|
||
| # check for dependencies | ||
| missing_packages="" | ||
| for dep in curl jq; do | ||
| if ! which $dep > /dev/null; then | ||
| missing_packages+=" $dep" | ||
| fi | ||
| done | ||
|
|
||
| if [ ! -z "${missing_packages}" ]; then | ||
| echo "Error: Requirement(s) not satisfied: ${missing_packages}" | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [[ ${build_version} =~ ^([0-9]+)\.([0-9]+).?([0-9]*)$ ]]; then | ||
| build_major=${BASH_REMATCH[1]} | ||
| build_minor=${BASH_REMATCH[2]} | ||
| build_patch=${BASH_REMATCH[3]} | ||
| else | ||
| echo "Error: The provided version ${build_version} does not have the expected format." | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [ -z ${build_patch} ]; then | ||
| build_patch=0 | ||
| fi | ||
|
|
||
| tag_update_flag=0 | ||
| # possible values of the tag_update flag: | ||
| # 0 = no additional tags to be set | ||
| # (this applies just in case of builds for older releases while a newer minor or minor.patch is already available) | ||
| # 1 = add the major.minor tag | ||
| # (this applies, if the build is for is just for a new patch version) | ||
| # 2 = add the tags major and major.minor | ||
| # (this applies if the build is for a version equal to or greater than the latest minor of a major) | ||
| # 3 = add the tags latest in addition to the tags from #2 | ||
| # (this applies if the build with the major, minor and patch version being equal to or greater than the currently latest version | ||
| # or if the build is for a completely new st2 component) | ||
|
|
||
| # check if there are already images for the given component available at dockerhub | ||
| dockerhub_registry_status_code=$(curl -sfL -o /dev/null -w "%{http_code}" https://registry.hub.docker.com/v1/repositories/stackstorm/${component}/tags) | ||
|
|
||
| if [ ${dockerhub_registry_status_code} -eq 404 ]; then | ||
| # there is no repository stackstorm/${component} available at dockerhub -> this build is for a new st2 component | ||
| tag_update_flag=3 | ||
| exit | ||
| elif [ ${dockerhub_registry_status_code} -ne 200 ]; then | ||
| echo "Error: Unexpected HTTP statuscode for https://registry.hub.docker.com/v1/repositories/stackstorm/${component}/tags: ${dockerhub_registry_status_code}" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # dockerhub lists the tags in ascending order. 1st object = lowest tag; last object = highest tag or latest | ||
| docker_tags_json=$(curl -s https://registry.hub.docker.com/v1/repositories/stackstorm/${component}/tags) | ||
| readarray -t available_releases < <(echo $docker_tags_json | jq -r '.[] | select((.name | endswith("dev") | not) and (.name=="latest" | not)).name' | sort) | ||
|
|
||
| # sort returns the list with the highest tag or "latest" as last item | ||
| # so change the value below to -2 when introducing the tag latest to get i.e. 3.3.0 instead of latest | ||
| latest_release=${available_releases[-1]} | ||
| latest_release_array=(${latest_release//\./ }) | ||
|
|
||
| latest_major=${latest_release_array[0]} | ||
| latest_minor=${latest_release_array[1]} | ||
|
|
||
| # validate the value stored as latest_release | ||
| if [[ ! ${latest_release} =~ ^([0-9]+)\.([0-9]+).?([0-9]*)$ ]]; then | ||
| echo "Error: Unexpected error. The latest release ${latest_release} does not match the expected format." | ||
| fi | ||
|
|
||
| if [ ${build_version} == ${latest_release} ]; then | ||
| # building a release of the latest st2 version | ||
| tag_update_flag=3 | ||
| else | ||
| # building a release for a st2 version that does not match the latest version | ||
| if [ ${build_major} -le ${latest_major} ]; then | ||
|
Member
There was a problem hiding this comment. Here is what I got when running the script locally:
Contributor
Author
There was a problem hiding this comment. Ofc, that shouldn't happen. I'll take a look at it tomorrow evening latest and try to provide a fix on the weekend. Will also see how this passed my own tests then.
Contributor
Author
There was a problem hiding this comment. Oh ok, I see how this happened. It looks like there are images at Dockerhub that do already have the latest tag and this one is now provided as the last item in the list (and no longer the actual latest release i.e. 3.3.0). I'll address that shortly.
Contributor
Author
There was a problem hiding this comment. Ok, this is fixed now. The main issue was the one I already described above: the script expected a release version like 3.3.0 and not The more important point is your 2nd one: This was a glidge in the script due to the "wonderful" (sorry, don't find an appropriate word for it right now :) ) error handling in bash. What happend is that the if clause failed due to to: So the worst thing that could happen now if parsing of a returned tag leads to an unwanted result is the error below: So the script returns 0 and no tag would be updated. I'll see if there is a more elegant solution to address this and even avoid (=handle) the 2 error messages. |
||
| # building a release for an older major | ||
| readarray -t build_version_major_minor_matching_releases < <(echo $docker_tags_json | jq -r '.[] | select(.name | endswith("dev") | not) | select(.name | startswith("'"${build_major}.${build_minor}"'")).name') | ||
winem marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| if [ ${#build_version_major_minor_matching_releases[@]} -ge 1 ]; then | ||
| # at least one version matching the current builds major and minor version is available | ||
| latest_build_version_major_minor_matching_release=${build_version_major_minor_matching_releases[-1]} | ||
| latest_build_version_major_minor_matching_release_array=(${latest_build_version_major_minor_matching_release//\./ }) | ||
| latest_build_version_major_minor_matching_major=${latest_build_version_major_minor_matching_release_array[0]} | ||
| latest_build_version_major_minor_matching_minor=${latest_build_version_major_minor_matching_release_array[1]} | ||
| latest_build_version_major_minor_matching_patch=${latest_build_version_major_minor_matching_release_array[2]} | ||
| if [ ${build_minor} -eq ${latest_minor} ] && \ | ||
| [ ${build_minor} -eq ${latest_build_version_major_minor_matching_minor} ] && \ | ||
| [ ${build_patch} -ge ${latest_build_version_major_minor_matching_patch} ]; then | ||
| # building a release for a new or updated patch version of the current major.minor version | ||
| tag_update_flag=3 | ||
| elif [ ${build_minor} -eq ${latest_build_version_major_minor_matching_minor} ] && \ | ||
| [ ${build_patch} -ge ${latest_build_version_major_minor_matching_patch} ]; then | ||
| # building a release for a new or updated patch version of an old major.minor version | ||
| readarray -t build_version_major_matching_releases < <(echo $docker_tags_json | jq -r '.[] | select(.name | endswith("dev") | not) | select(.name | startswith("'"${build_major}"'")).name') | ||
| latest_build_version_major_matching_release=${build_version_major_matching_releases[-1]} | ||
| latest_build_version_major_matching_release_array=(${latest_build_version_major_matching_release//\./ }) | ||
| latest_build_version_major_matching_minor=${latest_build_version_major_matching_release_array[1]} | ||
| if [ ${build_minor} -ge ${latest_build_version_major_matching_minor} ]; then | ||
| # building a release for a new or updated patch version of the latest or a new minor version of the major release | ||
| tag_update_flag=2 | ||
| else | ||
| # building a release for an older minor version of the major release | ||
| tag_update_flag=1 | ||
| fi | ||
| elif [ ${build_minor} -lt ${latest_minor} ] && [ ${build_patch} -ge ${latest_build_version_major_minor_matching_patch} ]; then | ||
| # building a patch release for an older minor version | ||
| tag_update_flag=1 | ||
| fi | ||
| elif [ ${build_minor} -gt ${latest_minor} ]; then | ||
| # building a release of a new minor version of the current or an old major version | ||
| if [ ${build_major} -eq ${latest_major} ]; then | ||
| # building a release of a new minor for the latest major version | ||
| tag_update_flag=3 | ||
| else | ||
| # building a release for a new minor of an old major version | ||
| tag_update_flag=2 | ||
| fi | ||
| #else | ||
| # # building a release for an old, unreleased major version of st2 | ||
| # tag_update_flag=2 | ||
| fi | ||
| elif [ ${build_major} -gt ${latest_major} ]; then | ||
| # building a release for a new major version | ||
| tag_update_flag=3 | ||
| fi | ||
| fi | ||
|
|
||
| echo $tag_update_flag | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think would be best to stop the entire process at early stage once we understood that script failed instead of waiting for the entire Docker build.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now all failures in the script should be handled and return a message with the "Error:"-Prefix and the makefile checks the output for the Error-Prefix and fails before building any images.
Another idea was to introduce another return value like -1 in case of an error but then I wouldn't know how to show the error message in the CI logs. If you have an idea to show the output from the script in the CI log please let me know.
I just pushed a Makefile with 3.3.0 and will change it back to 3.4dev before merging this branch. That's just because the determine tags script is skipped if we build a *dev version and not a release.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Depending on "Error" message as a failure indicator adds some knowledge overhead in requirements that could be broken in the future. What if 3rd Maintainer comes, edit the
determine_needed_tags.shscript and doesn't add "Error" message for the failure case.We can at least reinforce it with
determine_needed_tags.sh || echo Errorto fit your solution.