Skip to content

Add templates for required sysctls max_map_count and arp_announce#3440

Merged
openshift-merge-robot merged 1 commit into
openshift:masterfrom
dagrayvid:functional-sysctls
Dec 6, 2022
Merged

Add templates for required sysctls max_map_count and arp_announce#3440
openshift-merge-robot merged 1 commit into
openshift:masterfrom
dagrayvid:functional-sysctls

Conversation

@dagrayvid
Copy link
Copy Markdown
Contributor

- What I did
Add two files to templates/common/_base/files for setting sysctls vm.max_map_count and net.ipv4.all.arp_announce.

In OpenShift 4.13 we are planning to make the Node Tuning Operator an optional feature (composable OCP). Two sysctls currently set by NTO are needed for functional reasons, and should be set even in clusters where NTO is disabled.

vm.max_map_count=262144 is needed for OpenShift Logging to work (ElasticSearch), see RH BZ#1793714.
net.ipv4.conf.all.arp_announce=2 is needed for the OpenShift SDN (see RH BZ#1758552).

NTO will remain enabled by default and will continue to apply recommended default tunings to improve scalability and performance, while providing the option for users to apply custom TuneD profiles.

- How to verify it
Ensure that /etc/sysctl.d/arp.conf and /etc/sysctl.d/vm-max-map.conf are created, and that the sysctl values are set correctly even when these sysctls are not set by NTO.

- Description for the changelog
Add templates for required sysctls max_map_count and arp_announce

/cc @jmencak

@openshift-ci openshift-ci Bot requested a review from jmencak December 1, 2022 21:14
@dagrayvid
Copy link
Copy Markdown
Contributor Author

/retest

Copy link
Copy Markdown
Contributor

@jmencak jmencak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR, David. The changes look good to me. Personally, I'd add a one-liner comment inside each of the new template files why the setting is needed with a link to the respective BZs though, so that it is immediately clear why the setting is needed without looking at the git history.

In OpenShift 4.13 we are planning to make the Node Tuning Operator
an optional feature (composable OCP). Two sysctls currently set by
NTO are needed for functional reasons, and should be set even in
clusters where NTO is disabled.

vm.max_map_count=262144 is needed for OpenShift Logging to work
(ElasticSearch), see RH BZ#1793714.
net.ipv4.conf.all.arp_announce=2 is needed for the OpenShift SDN
(see RH BZ#1758552).

NTO will remain enabled by default and will continue to apply
recommended default tunings to improve scalability and performance,
while providing the option for users to apply custom TuneD profiles.
@dagrayvid
Copy link
Copy Markdown
Contributor Author

Thank you for the PR, David. The changes look good to me. Personally, I'd add a one-liner comment inside each of the new template files why the setting is needed with a link to the respective BZs though, so that it is immediately clear why the setting is needed without looking at the git history.

Thanks Jiri, I did this. I added the comments into the .conf files themselves, so that someone looking at the host OS will see the explanations.

Copy link
Copy Markdown
Contributor

@jmencak jmencak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the changes, David.
/lgtm
Needs approvals from the MCO team though.
/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 2, 2022
@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Dec 2, 2022
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Dec 2, 2022

@dagrayvid: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-e2e-aws aa77497 link false /test okd-scos-e2e-aws
ci/prow/e2e-hypershift aa77497 link false /test e2e-hypershift

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@dagrayvid
Copy link
Copy Markdown
Contributor Author

/assign @jkyros

Copy link
Copy Markdown
Contributor

@yuqi-zhang yuqi-zhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense, just curious:

  1. is it possible for a cluster to go from having these unset to having them set (when upgrading from 4.12->4.13), and does that affect anything
  2. for existing clusters with NTO as well, these will be presumably defined twice (which isn't an issue)
  3. in 4.13, would you be able to remove NTO from an upgraded cluster? would that also delete associated MCs? Or is this only something you can set during install time?

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Dec 6, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dagrayvid, jmencak, yuqi-zhang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 6, 2022
@jmencak
Copy link
Copy Markdown
Contributor

jmencak commented Dec 6, 2022

One issue I see is that people using NTO to set these tunables (vm.max_map_count in particular) to something higher (say 524288) will see a regression after merging this, because settings in /etc/sysctl.d take precedence over NTO's settings unless NTO is explicitly configured to do that (it is not by default). So if merged, this information needs to be in the 4.13 release notes + perhaps a brief note how to configure the settings correctly.

@dagrayvid
Copy link
Copy Markdown
Contributor Author

  1. Is it possible for a cluster to go from having these unset to having them set (when upgrading from 4.12->4.13), and does that affect anything

In 4.12, NTO will be enabled and will by-default set these itself via the containerized TuneD daemon (operand). The only issue here is the one Jiri mentioned. If a customer is using a custom TuneD profile that set's these to some other values, it will look like it no longer takes effect on 4.13, unless they have configured TuneD to override the settings from /etc/sysctl.d, so this should be noted in the release notes.

  1. for existing clusters with NTO as well, these will be presumably defined twice (which isn't an issue)

Correct, should not be an issue.

  1. in 4.13, would you be able to remove NTO from an upgraded cluster? would that also delete associated MCs? Or is this only something you can set during install time?

My understanding is that you can only disable features like this at install time. You can enabled disabled-features after install, but you can't disable the enabled-features later. Regardless, I don't think this would have an impact, NTO sets sysctls via TuneD, not via MC's.

@dagrayvid
Copy link
Copy Markdown
Contributor Author

/unhold

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 6, 2022
@yuqi-zhang
Copy link
Copy Markdown
Contributor

Sounds good, thanks for the clarifications!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants