Refactor controller with pkg runtime helpers by phillebaba · Pull Request #342 · fluxcd/helm-controller

phillebaba · 2021-10-19T08:48:35Z

Fixes #322

HelmRelease should implement the interfaces from conditions, and use this package to work with conditions.
HelmReleaseReconciler should implement the helpers from controller, and use these to record Events and Metrics.
HelmReleaseReconciler should exit early without recording any status when .spec.suspend is defined.
The patch package should be implemented to (safely) patch resources without running into conflicts.
The kstatus Reconciling and Stalled conditions should be properly implemented to provide better signals.

phillebaba · 2021-10-27T12:29:57Z

-	// +kubebuilder:validation:Enum=ChartVersion;Revision
-	// +kubebuilder:default:=ChartVersion
-	// +optional
-	ReconcileStrategy string `json:"reconcileStrategy,omitempty"`


This was removed from the HelmChart so I assume that it should be removed from here also.

My guess is that the answer is yes.

Missed this comment for some reason :-S

No, it was introduced to the HelmChart resource in a later version than the reconcilers-dev branch. So you don't have access to the API against this version, but it will return.

Ok good, will keep this open and loop back when it becomes relevant.

phillebaba · 2021-10-27T12:31:15Z

+			),
+			conditions.WithNegativePolarityConditions(
+				v2.RemediatedCondition,
+			),


All of this is probably wrong, need to go back and read up on conditions.

This should contain a list of all conditions that have an influence on the end result of Ready. What defines if something is (or isn't) a negative polarity is if the condition highlights normal vs abnormal behavior, a condition type can't be both negative and positive (and should thus only be listed once).

Examples:

Ready == True is "normal behavior", as the condition is present and True if everything is operating as expected.

Stalled == True is "abnormal behavior" as the condition is present and True if an abnormality occurs, but removed from the resource once resolved.

RemediatedCondition is a negative polarity condition, as it being present indicates an error and should result in an inverse of the True and return a Ready == False.

So for now it looks right, but I will have to come back in a later PR and make changes to the different conditions.

phillebaba · 2021-10-27T18:03:27Z

@hiddeco I am getting a hang of most things. Things seem to work in a full deployment now. The most confusing thing right now is the different conditions and the preferred methods. For example the differences between using released conditon and ready condition.

Need to work some of these things to make sure that we are not breaking existing functionality.

phillebaba · 2021-11-02T15:27:58Z

-			ValuesFile:        template.Spec.ValuesFile,
+			Interval:    template.GetInterval(hr.Spec.Interval),
+			ValuesFiles: template.Spec.ValuesFiles,
+			ValuesFile:  template.Spec.ValuesFile,


Should Suspend and AccessFrom be added to here?

Introduction of suspend would be an idea, but also raises a question: if a user decides to add both fields, the one of the chart won't be reconciled. Is this expected behavior, or an awkward result of design decisions?

For AccessFrom, @stefanprodan will likely be able to tell more.

I think the suspend feature should be in its own PR to keep things simple. But yes the question about setting both fields is an important one. Is it due to legacy that we have both field options or is there some other reason behind this? My suggestion for now is to just throw an error if both are set.

Given that the controller generates the HelmChart, AccessFrom should be set to allow the namespace where the HelmRelease is. Sadly it's impossible to to restrict the access to a single namespace on Kubernetes < 1.21 since the kubernetes.io/metadata.name label was aded in 1.21. Given this I would set this to allow all.

phillebaba · 2021-11-02T15:30:24Z

-		r.recordReadiness(ctx, hr)
-	}
+	// TODO: Progressing is set immediately on reconcile so is it necessary to set it here?
+	hr, _ = v2.HelmReleaseAttempted(hr, revision, releaseRevision, valuesChecksum)


Should hasNewState be used here?

To keep most of the logic (that should be rewritten as well as noted in (an) other issue(s)) intact I would maintain it for now.

Would the easiest solution be to skip the status patching and only implement the failure counter reset?

hiddeco · 2021-11-08T11:54:36Z

-			ValuesFile:        template.Spec.ValuesFile,
+			Interval:    template.GetInterval(hr.Spec.Interval),
+			ValuesFiles: template.Spec.ValuesFiles,
+			ValuesFile:  template.Spec.ValuesFile,


Introduction of suspend would be an idea, but also raises a question: if a user decides to add both fields, the one of the chart won't be reconciled. Is this expected behavior, or an awkward result of design decisions?

For AccessFrom, @stefanprodan will likely be able to tell more.

hiddeco · 2021-11-08T11:59:07Z

+			),
+			conditions.WithNegativePolarityConditions(
+				v2.RemediatedCondition,
+			),


This should contain a list of all conditions that have an influence on the end result of Ready. What defines if something is (or isn't) a negative polarity is if the condition highlights normal vs abnormal behavior, a condition type can't be both negative and positive (and should thus only be listed once).

Examples:

Ready == True is "normal behavior", as the condition is present and True if everything is operating as expected.

Stalled == True is "abnormal behavior" as the condition is present and True if an abnormality occurs, but removed from the resource once resolved.

RemediatedCondition is a negative polarity condition, as it being present indicates an error and should result in an inverse of the True and return a Ready == False.

hiddeco · 2021-11-08T12:07:28Z

-		r.recordReadiness(ctx, hr)
-	}
+	// TODO: Progressing is set immediately on reconcile so is it necessary to set it here?
+	hr, _ = v2.HelmReleaseAttempted(hr, revision, releaseRevision, valuesChecksum)


To keep most of the logic (that should be rewritten as well as noted in (an) other issue(s)) intact I would maintain it for now.

Signed-off-by: Philip Laine <philip.laine@xenit.se>

darkowlzz · 2021-12-02T13:56:46Z

-	if reconcileErr != nil {
-		r.event(ctx, hr, hc.GetArtifact().Revision, events.EventSeverityError,
-			fmt.Sprintf("reconciliation failed: %s", reconcileErr.Error()))
+	if result, err := r.reconcileRelease(ctx, obj, chart, values); reconcileErr != nil {


I've been investigating the reason for the test failure in CI and found that because of reconcileErr here, the error from reconcileRelease() is always ignored. So the install-fail test doesn't gets expected result.

Suggested change

if result, err := r.reconcileRelease(ctx, obj, chart, values); reconcileErr != nil {

if result, err := r.reconcileRelease(ctx, obj, chart, values); err != nil {

This may not resolve the CI failures, there's more to it. Investigating...

This should fix most of the CI failures, but I found some more conditionals related issues. Will comment them inline.

darkowlzz

I tried reproducing the e2e tests that are failing in the CI and left comments about what I did to make it work. Also left a few comments around the status conditions usage. Hope those comments will make it clear some of the conditionals usage patterns.

darkowlzz · 2021-12-02T22:29:56Z

-		r.event(ctx, hr, hr.Status.LastAttemptedRevision, events.EventSeverityError, msg)
-		return v2.HelmReleaseNotReady(hr, v2.ArtifactFailedReason, msg), ctrl.Result{Requeue: true}, reconcileErr
+		obj.IncrementFailureCounter()
+		conditions.MarkFalse(obj, meta.ReadyCondition, v2.ArtifactFailedReason, "Chart reconcilliation failed: %s", reconcileErr.Error())


Since ready condition is calculated as a summary based on other conditions, it'd be better to mark ReleasedCondition false here. Setting ready condition here gets overwritten in the Reconcile() when ready condition is set as a summary of other conditions.

darkowlzz · 2021-12-02T22:31:08Z

 		msg := fmt.Sprintf("HelmChart '%s/%s' is not ready", hc.GetNamespace(), hc.GetName())
-		r.event(ctx, hr, hr.Status.LastAttemptedRevision, events.EventSeverityInfo, msg)
+		obj.IncrementFailureCounter()
+		conditions.MarkFalse(obj, meta.ReadyCondition, v2.ArtifactFailedReason, msg)
