Skip to content

controller: Fix "changed" logic to account for nodeReady changes#718

Closed
cgwalters wants to merge 2 commits intoopenshift:masterfrom
cgwalters:fix-nodechanged
Closed

controller: Fix "changed" logic to account for nodeReady changes#718
cgwalters wants to merge 2 commits intoopenshift:masterfrom
cgwalters:fix-nodechanged

Conversation

@cgwalters
Copy link
Copy Markdown
Member

The nodeChanged logic was not accounting for the fact that
the controller takes readiness into account too. However, rather
than change that function, let's just remove it and extend our
logic below that which was also effectively doing change detection
so it could log it.

This also removes the logging from nodeReady() which got very
noisy in status; we now consistently log just changes in the node
controller.

However, the change detection logic was also implicitly ignoring
nodes which didn't appear to be managed by the MCD at all - think
Windows nodes. Let's explicitly skip nodes that don't have a currentConfig
annotation.

@openshift-ci-robot openshift-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels May 8, 2019
Comment thread pkg/controller/node/status.go Outdated
Comment thread pkg/controller/node/node_controller.go Outdated
@openshift-ci-robot openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 8, 2019
@runcom
Copy link
Copy Markdown
Member

runcom commented May 8, 2019

I'm a bit afraid of changing this code now, but looks sane to me and we need to fix #701 anyway

/lgtm

@kikisdeliveryservice @LorbusChris ptal as well

@openshift-ci-robot openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels May 8, 2019
cgwalters added 2 commits May 8, 2019 15:17
Continuing on the quest to make the controller logs more useful.
The `nodeChanged` logic was not accounting for the fact that
the controller takes readiness into account too.  However, rather
than change that function, let's just remove it and extend our
logic below that which was also effectively doing change detection
so it could log it.

This also removes the logging from `nodeReady()` which got very
noisy in status; we now consistently log just changes in the node
controller.

However, the change detection logic was also implicitly ignoring
nodes which didn't appear to be managed by the MCD at all - think
Windows nodes.  Let's explicitly skip nodes that don't have a `currentConfig`
annotation.
@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label May 8, 2019
@cgwalters
Copy link
Copy Markdown
Member Author

OK this one is rebased 🏄‍♂️ on master which has #719 now.

@cgwalters
Copy link
Copy Markdown
Member Author

Ok this is rebased 🏄‍♂️ on master which as #719 now.

Copy link
Copy Markdown
Contributor

@kikisdeliveryservice kikisdeliveryservice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a question and a few notes (no updates required).

Comment thread pkg/controller/node/status.go
if cond.Type == corev1.NodeReady && cond.Status != corev1.ConditionTrue {
glog.Infof("Node %s is reporting NotReady", node.Name)
return false
return fmt.Errorf("Node %s is reporting NotReady", node.Name)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: these should be lowercase error messages. we can fix later.

Comment thread pkg/controller/node/status.go
oldReady := isNodeReady(oldNode)
newReady := isNodeReady(curNode)
var changed bool
oldReadyErr := checkNodeReady(oldNode)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

later: we could make this a small function instead of repeating code

@kikisdeliveryservice
Copy link
Copy Markdown
Contributor

went through this and LGTM

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label May 8, 2019
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ashcrow, cgwalters, kikisdeliveryservice, runcom

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [ashcrow,cgwalters,kikisdeliveryservice,runcom]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@runcom
Copy link
Copy Markdown
Member

runcom commented May 8, 2019

/hold

for beta I guess

@cgwalters
Copy link
Copy Markdown
Member Author

This was merged as part of #701

@cgwalters cgwalters closed this May 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants