Skip to content

Change CI workflows to release tagged container images for Dataverse#11477

Merged
ofahimIQSS merged 66 commits intoIQSS:developfrom
gdcc:10618-app-image-tags
Jun 9, 2025
Merged

Change CI workflows to release tagged container images for Dataverse#11477
ofahimIQSS merged 66 commits intoIQSS:developfrom
gdcc:10618-app-image-tags

Conversation

@poikilotherm
Copy link
Contributor

@poikilotherm poikilotherm commented May 8, 2025

What this PR does / why we need it:
People demand container images of Dataverse with tags of released versions of Dataverse.
This PR will meet their demands.

Open TODOs before this is ready for review:

  • Add a README for src/backports
  • Re-enable the "run at IQSS only" guards again once review is done
  • Add label in app image with reference to base image tag (not unless we can figure out Cannot use properties from build-helper Maven Plugin fabric8io/docker-maven-plugin#1865)
  • Update docs on supported images, also add note about scheduled maintenance
  • Update devguide/makerelease page about container images
  • Preview images for base image -> new issue, new dev cycle.
  • Add release note snippet
  • Configbaker: add maintenance script, for now using rolling tags only (very low change rate here, not many deps installed)
  • Configbaker: pin down the versions of software installed in the image to avoid nasty surprises
  • Configbaker: add backports
  • Add Renovate Bot config to update the pinned dependencies in containers every Sunday

Which issue(s) this PR closes:

Special notes for your reviewer:
It needs to be decided if not running the Maven Unit Test workflow on push to master and not on new tags.
Otherwise, this is only touching container stuff, low risk for anyone else.

Suggestions on how to test this:

  1. Take a look at the action runs over at https://github.com/gdcc/wip-dataverse-base-image/actions/workflows/container_maintenance.yml. You can also dispatch a workflow run from the dropdown, just make sure to set dry-run mode.
  2. You can also execute the CI scripts locally, just use DRY_RUN=1 to avoid building anything or DAMP_RUN=1 to see Maven build Docker images but not push them to the Hub. Make sure to call the scripts from the repo's root via ./.github/workflows/scripts/containers/maintain-YYY.sh <branch names you want included>

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
None.

Is there a release notes update needed for this change?:
🔋 included

Additional documentation:
Preview docs at https://dataverse-guide--11477.org.readthedocs.build/en/11477/container/index.html

@poikilotherm poikilotherm self-assigned this May 8, 2025
@poikilotherm poikilotherm added Component: Code Infrastructure formerly "Feature: Code Infrastructure" Feature: Developer Guide Component: Containers Anything related to cloudy Dataverse, shipped in containers. labels May 8, 2025
@poikilotherm poikilotherm added Feature: Container Guide Release Issue is related to making a software release (e.g., Dataverse) labels May 8, 2025
@pdurbin pdurbin moved this to In Progress 💻 in IQSS Dataverse Project May 8, 2025
@poikilotherm poikilotherm force-pushed the 10618-app-image-tags branch from 40c2d0e to 53843cc Compare May 15, 2025 10:12
@cmbz cmbz added FY25 Sprint 24 FY25 Sprint 24 (2025-05-21 - 2025-06-04) and removed FY25 Sprint 24 FY25 Sprint 24 (2025-05-21 - 2025-06-04) labels May 22, 2025
@poikilotherm poikilotherm force-pushed the 10618-app-image-tags branch from da51a39 to bb0262a Compare June 2, 2025 08:44
@coveralls
Copy link

coveralls commented Jun 2, 2025

Coverage Status

coverage: 23.146% (+0.005%) from 23.141%
when pulling fafd871 on gdcc:10618-app-image-tags
into 7a3964b on IQSS:develop.

@poikilotherm poikilotherm force-pushed the 10618-app-image-tags branch 3 times, most recently from 5861f79 to db86eb1 Compare June 2, 2025 14:07
@poikilotherm poikilotherm force-pushed the 10618-app-image-tags branch 2 times, most recently from aa8e910 to 09518af Compare June 2, 2025 15:20
@pdurbin pdurbin added this to the 6.7 milestone Jun 2, 2025
For the base image we didn't use a parent image which has a namespace. If the given image has a namespace, it needs to be removed from the repo string to create the tags lookup URL.
Done because we want to have a similar script for other images as well.
We will add more jobs, so make the job's name more speaking, referring to what happens inside
This is necessary so we can have the maintenance jobs only work on their respective parts and not always do both.

Otherwise the configbaker image would have many more fixed tags than necessary (the same as the app image has), confusing people because there were no changes.
We will need it to have the version number ready for the maintenance job.
At the same time, leaving a note that we will _not_ use it for the default tag we use during development.

If we'd use the version there, too, it would break many scripts that now can rely on the single rolling tag of "unstable", as we'd need to change the tags everywhere for a new version.
@cmbz cmbz added the FY25 Sprint 25 FY25 Sprint 25 (2025-06-04 - 2025-06-18) label Jun 4, 2025
Removed temporary testing condition for `gdcc/wip-base-image` and reinstated the `if` condition to restrict workflow runs to the upstream `IQSS` repository.
Introduced documentation detailing the structure, purpose, and usage of the `src/backports` directory to support consistent patch management across multiple Dataverse release versions.
@poikilotherm
Copy link
Contributor Author

@pdurbin as requested, I added a README to src/backports

Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm relying on the testing @poikilotherm did but this looks good to me. Approved.

@github-project-automation github-project-automation bot moved this from In Review 🔎 to Ready for QA ⏩ in IQSS Dataverse Project Jun 4, 2025
@pdurbin pdurbin mentioned this pull request Jun 5, 2025
4 tasks
@ofahimIQSS ofahimIQSS self-assigned this Jun 9, 2025
@ofahimIQSS ofahimIQSS moved this from Ready for QA ⏩ to QA ✅ in IQSS Dataverse Project Jun 9, 2025
@ofahimIQSS
Copy link
Contributor

I reran the failing continuous integration job - it looks like it failed again: https://jenkins.dataverse.org/blue/organizations/jenkins/IQSS-Dataverse-Develop-PR/detail/PR-11477/30/pipeline

@ofahimIQSS
Copy link
Contributor

Looks good on my end. Merging.

@ofahimIQSS ofahimIQSS merged commit 74d4405 into IQSS:develop Jun 9, 2025
17 of 18 checks passed
@github-project-automation github-project-automation bot moved this from QA ✅ to Merged 🚀 in IQSS Dataverse Project Jun 9, 2025
@ofahimIQSS ofahimIQSS removed their assignment Jun 9, 2025
@pdurbin pdurbin moved this from Merged 🚀 to Done 🧹 in IQSS Dataverse Project Jun 16, 2025
@cmbz cmbz added the FY26 Sprint 4 FY26 Sprint 4 (2025-08-13 - 2025-08-27) label Aug 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Code Infrastructure formerly "Feature: Code Infrastructure" Component: Containers Anything related to cloudy Dataverse, shipped in containers. Feature: Container Guide Feature: Developer Guide FY25 Sprint 24 FY25 Sprint 24 (2025-05-21 - 2025-06-04) FY25 Sprint 25 FY25 Sprint 25 (2025-06-04 - 2025-06-18) FY26 Sprint 4 FY26 Sprint 4 (2025-08-13 - 2025-08-27) Release Issue is related to making a software release (e.g., Dataverse)

Projects

Status: Important
Status: Done 🧹

Development

Successfully merging this pull request may close these issues.

Tags for application container images

5 participants