bumped MUO version in const.go #4499

ehvs · 2025-12-03T13:03:15Z

Which issue this PR addresses:

https://issues.redhat.com/browse/ARO-22964
Side-car PR mandatory for #4494

What does it fixes:

It sets the operator version to match the new one set in the mirror on #4494
It removes the validation based on a string for boringcrypto,strictfipsruntime, because of OCP upstream update PR here/OSD-29374: Drop unsupported strictfipsruntime GOEXPERIMENT for Go 1.23, and relies on boringcrypto
It updates the object role to have the updated parameters as per upstream change, Enabling Lease based leader-election for MUO openshift/managed-upgrade-operator#517

Test plan for issue:

e2e
✅ Compatibility test Tested in a dev cluster calling AdminUpdate.

Created a local dev rp with master branch (without the changes)
Switched to my dev branch and ran the RP
Called AdminUpdate with curl command
Results:
a. The ARO_IMAGE for the operator was updated successfully
b. The MUO image version was updated successfully
c. The role object was updated.

✅ New cluster Built a custom image with the changes and it ran successful.
✅ Unit testing

# go test -v ./pkg/operator/controllers/muo
=== RUN   TestMUOReconciler
=== RUN   TestMUOReconciler/disabled
=== RUN   TestMUOReconciler/managed
=== RUN   TestMUOReconciler/managed,_no_pullspec_(uses_default)
=== RUN   TestMUOReconciler/managed,_MUO_does_not_become_ready
=== RUN   TestMUOReconciler/managed,_could_not_parse_cluster_version_fails
=== RUN   TestMUOReconciler/managed,_CreateOrUpdate()_fails
=== RUN   TestMUOReconciler/managed=false_(removal)
=== RUN   TestMUOReconciler/managed=false_(removal),_Remove()_fails
=== RUN   TestMUOReconciler/managed=blank_(no_action)
--- PASS: TestMUOReconciler (0.00s)
    --- PASS: TestMUOReconciler/disabled (0.00s)
    --- PASS: TestMUOReconciler/managed (0.00s)
    --- PASS: TestMUOReconciler/managed,_no_pullspec_(uses_default) (0.00s)
    --- PASS: TestMUOReconciler/managed,_MUO_does_not_become_ready (0.00s)
    --- PASS: TestMUOReconciler/managed,_could_not_parse_cluster_version_fails (0.00s)
    --- PASS: TestMUOReconciler/managed,_CreateOrUpdate()_fails (0.00s)
    --- PASS: TestMUOReconciler/managed=false_(removal) (0.00s)
    --- PASS: TestMUOReconciler/managed=false_(removal),_Remove()_fails (0.00s)
    --- PASS: TestMUOReconciler/managed=blank_(no_action) (0.00s)
=== RUN   TestDeployCreateOrUpdateCorrectKinds
--- PASS: TestDeployCreateOrUpdateCorrectKinds (0.00s)
=== RUN   TestDeployConfig
--- PASS: TestDeployConfig (0.00s)
PASS
ok      github.com/Azure/ARO-RP/pkg/operator/controllers/muo    0.023s

Other logs

# oc get pods
NAME                                        READY   STATUS    RESTARTS   AGE
managed-upgrade-operator-5cc48fc9f7-c944h   1/1     Running   0          21m


ts=2025-12-11T12:30:42.19783184Z level=info msg="Starting EventSource" controller=upgradeconfig controllerGroup=upgrade.managed.openshift.io controllerKind=UpgradeConfig source="kind source: *v1alpha1.UpgradeConfig"
ts=2025-12-11T12:30:42.19787924Z level=info msg="Starting EventSource" controller=machineconfigpool controllerGroup=machineconfiguration.openshift.io controllerKind=MachineConfigPool source="kind source: *v1.MachineConfigPool"
ts=2025-12-11T12:30:42.19801364Z level=info msg="Starting EventSource" controller=node controllerGroup= controllerKind=Node source="kind source: *v1.Node"
ts=2025-12-11T12:30:42.385416628Z level=info msg="Starting Controller" controller=machineconfigpool controllerGroup=machineconfiguration.openshift.io controllerKind=MachineConfigPool
ts=2025-12-11T12:30:42.385442328Z level=info msg="Starting workers" controller=machineconfigpool controllerGroup=machineconfiguration.openshift.io controllerKind=MachineConfigPool workercount=1
ts=2025-12-11T12:30:42.385449628Z level=info msg="Starting Controller" controller=node controllerGroup= controllerKind=Node
ts=2025-12-11T12:30:42.385460228Z level=info msg="Starting workers" controller=node controllerGroup= controllerKind=Node workercount=1
ts=2025-12-11T12:30:42.385462728Z level=info msg="Starting Controller" controller=upgradeconfig controllerGroup=upgrade.managed.openshift.io controllerKind=UpgradeConfig
ts=2025-12-11T12:30:42.385469028Z level=info msg="Starting workers" controller=upgradeconfig controllerGroup=upgrade.managed.openshift.io controllerKind=UpgradeConfig workercount=1
ts=2025-12-11T12:30:42.385535928Z level=info logger=controller_machineconfigpool msg="Reconciling MachineConfigPool" Request.Namespace= Request.Name=worker

mrWinston

Looks good!

…pgrade-operator#517

ehvs · 2025-12-11T17:37:48Z

The error in e2e is:

run-e2e  |   [TIMEDOUT] ARO Operator - MUO Deployment [It] must be deployed by default with FIPS crypto mandated
run-e2e  |   /app/test/e2e/operator.go:435

This is a test that we are doing inspecting a string g.Expect(body).To(ContainSubstring(X:boringcrypto,strictfipsruntime))
When the logs on the new image does not have strictfipsruntime

ts=2025-12-11T12:19:48.086416145Z level=info logger=cmd msg="Operator Version: 1.0.0"
ts=2025-12-11T12:19:48.086432345Z level=info logger=cmd msg="Go Version: go1.24.9 X:boringcrypto"
ts=2025-12-11T12:19:48.086435745Z level=info logger=cmd msg="Go OS/Arch: linux/amd64"
ts=2025-12-11T12:19:48.086438245Z level=info logger=cmd msg="Version of operator-sdk: v1.21.0"

To fix this will need to fix the validation.
Breadcrumbs
openshift/boilerplate#516
openshift/managed-upgrade-operator#505

ehvs · 2025-12-11T20:02:01Z

MIWI failed, rerunning to check if is flake.

run-e2e  |   [FAIL] ARO Operator - dnsmasq [It] must respect the forceReconciliation flag and not update MachineConfigs by default
run-e2e  |   /app/test/e2e/operator.go:695
run-e2e  | 
run-e2e  | Ran 70 of 94 Specs in 5896.345 seconds
run-e2e  | FAIL! -- 69 Passed | 1 Failed | 5 Pending | 19 Skipped

RickARodriguez

This PR was to bump MUO, what does the update to the static resource affect in this instance? Will it be applied on future installs? Or will it get pushed to the operator on an admin update? And are there any concern about backwards comparability?

ehvs · 2025-12-12T10:59:33Z

This RP was to bump MUO, what does the update to the static resource affect in this instance? Will it be applied on future installs? Or will it get pushed to the operator on an admin update? And are there any concern about backwards comparability?

Good questions.

the static resource is required because of the change in upstream openshift/muo/#517. (I've updated the PR description)
It will be applied on future installs
Yes, The existing clusters will need an admin-update
My understanding is that there is no concerns for backwards compatibility because once the ARO operator is updated it will reconcile the whole thing and use the new version and settings.

ehvs · 2026-01-05T13:54:37Z

Updated the description to include all the testing made.

mociarain · 2026-01-05T14:05:33Z

test/e2e/operator.go

 		managedUpgradeOperatorDeployment = "managed-upgrade-operator"
 	)

-	It("must be deployed by default with FIPS crypto mandated", func(ctx context.Context) {


So, I get that this test is broken due to the update but is it true that the behavior this test outlined is no longer needed?

Yes, correct.

Can you go into some more detail? Any change related to FIPS makes me extra cautious

Our check first is a string match on a log message, which is not an ensurance.
The MUO image uses the operator boilerplate, that is actually controlled by the owners of the operator in boilerplate repository.
Currently MUO owners dropped the use of strictfipsruntime because it was giving errors to them, and they are now relying on boringcrypto (check PR description)
This operator is managed by ROSA SRE which is FedRamp compliant, meaning FIPS compliant.

ehvs requested review from bennerv, cadenmarchese, fahlmant, hawkowl, hlipsig, jharrington22, kimorris27, mociarain, mrWinston, pepedocs, rogbas, sankur-codes, tiguelu, tsatam, wanghaoran1988 and yjst2012 as code owners December 3, 2025 13:03

mrWinston previously approved these changes Dec 3, 2025

View reviewed changes

tiguelu previously approved these changes Dec 3, 2025

View reviewed changes

ehvs added the hold Hold label Dec 3, 2025

ehvs dismissed stale reviews from tiguelu and mrWinston via 4e0ef48 December 11, 2025 12:43

ehvs added 3 commits December 11, 2025 16:21

bumped MUO version in const.go

e7c3c1e

bumping to muo_controller_test

d2fa8cf

Added missing atributes to the role as per MUO PR openshift/managed-u…

f62a539

…pgrade-operator#517

ehvs force-pushed the ehvs/ARO-22964-2 branch from 4e0ef48 to f62a539 Compare December 11, 2025 15:23

Added new line to YAML to pass mega-linter test

807ad97

ehvs added 2 commits December 11, 2025 18:43

upon Go 1.24 FIPS is working with boringcrypto

a345f07

new line to yaml - fixing trailing spaces

22802a7

ehvs added ready-for-review and removed hold Hold labels Dec 12, 2025

RickARodriguez reviewed Dec 12, 2025

View reviewed changes

mociarain reviewed Jan 5, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bumped MUO version in const.go #4499

bumped MUO version in const.go #4499

ehvs commented Dec 3, 2025 •

edited

Loading

Uh oh!

mrWinston left a comment

Uh oh!

ehvs commented Dec 11, 2025

Uh oh!

ehvs commented Dec 11, 2025

Uh oh!

RickARodriguez left a comment •

edited

Loading

Uh oh!

ehvs commented Dec 12, 2025 •

edited

Loading

Uh oh!

ehvs commented Jan 5, 2026

Uh oh!

mociarain Jan 5, 2026

Uh oh!

ehvs Jan 5, 2026

Uh oh!

mociarain Jan 6, 2026

Uh oh!

ehvs Jan 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bumped MUO version in const.go #4499

Are you sure you want to change the base?

bumped MUO version in const.go #4499

Conversation

ehvs commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue this PR addresses:

What does it fixes:

Test plan for issue:

Other logs

Uh oh!

mrWinston left a comment

Choose a reason for hiding this comment

Uh oh!

ehvs commented Dec 11, 2025

Uh oh!

ehvs commented Dec 11, 2025

Uh oh!

RickARodriguez left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ehvs commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ehvs commented Jan 5, 2026

Uh oh!

mociarain Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

ehvs Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

mociarain Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

ehvs Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ehvs commented Dec 3, 2025 •

edited

Loading

RickARodriguez left a comment •

edited

Loading

ehvs commented Dec 12, 2025 •

edited

Loading