Bug 1847675: [4.4] mount /var/run/netns rslave in ovnkube by haircommander · Pull Request #669 · openshift/cluster-network-operator

haircommander · 2020-06-15T16:52:55Z

When trying to transfer cri-o to manage its network namespaces in openshift, we have run into problems with multus. Specifically we see the error:
2020-04-07T17:46:52Z [error] delegateAdd: error invoking DelegateAdd - "ovn-k8s-cni-overlay": error in getting result from AddNetwork: CNI request failed with status 400: '[openshift-dns/dns-default-98tll] failed to configure pod interface: failed to open netns "/var/run/netns/76716600-f7fd-462f-b7df-ae054dbd144e": unknown FS magic on "/var/run/netns/76716600-f7fd-462f-b7df-ae054dbd144e": 1021994
'

through testing, I've found the netns is definitely mounted as an nsfs and not tmpfs, so I suspect we are seeing containernetworking/plugins#69

to fix this, attempt mounting /var/run/netns as HostToContainer in ovnkube container

This is part 1 of 2 fixes for https://bugzilla.redhat.com/show_bug.cgi?id=1825339
the second will be a change in MCO

backport of #579

Signed-off-by: Peter Hunt pehunt@redhat.com

When trying to transfer cri-o to manage its network namespaces in openshift, we have run into problems with multus. Specifically we see the error: 2020-04-07T17:46:52Z [error] delegateAdd: error invoking DelegateAdd - "ovn-k8s-cni-overlay": error in getting result from AddNetwork: CNI request failed with status 400: '[openshift-dns/dns-default-98tll] failed to configure pod interface: failed to open netns "/var/run/netns/76716600-f7fd-462f-b7df-ae054dbd144e": unknown FS magic on "/var/run/netns/76716600-f7fd-462f-b7df-ae054dbd144e": 1021994 ' through testing, I've found the netns is definitely mounted as an nsfs and not tmpfs, so I suspect we are seeing containernetworking/plugins#69 to fix this, attempt mounting /var/run/netns as HostToContainer in ovnkube container Signed-off-by: Peter Hunt <pehunt@redhat.com>

openshift-ci-robot · 2020-06-15T16:52:56Z

@haircommander: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

mount /var/run/netns rslave in ovnkube

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

haircommander · 2020-06-15T18:48:41Z

/retest

haircommander · 2020-06-15T20:39:15Z

/test e2e-metal-ipi

haircommander · 2020-06-15T21:07:24Z

error: unable to push registry.svc.ci.openshift.org/ci-op-k28i61il/stable: failed to retrieve blob sha256:1ca95179947fa266f1a465167f76257b9b94160b155546852561bf3165ad303c: error parsing HTTP 408 response body: invalid character '<' looking for beginning of value: "<html><body><h1>408 Request Time-out</h1>\nYour browser didn't send a complete request in time.\n</body></html>\n"

/retest

pecameron

@haircommander do you also need this in ovnkube-master? Also, does this impact openshift-sdn?

danwinship · 2020-06-16T15:03:01Z

Should mention that this is a backport of #579, and it needs a bugzilla bug

danwinship · 2020-06-16T15:03:18Z

/lgtm

openshift-ci-robot · 2020-06-16T15:05:10Z

@haircommander: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

mount /var/run/netns rslave in ovnkube

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-06-16T15:05:10Z

@haircommander: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

[4.4] mount /var/run/netns rslave in ovnkube

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

haircommander · 2020-06-16T15:07:30Z

@pecameron already there

cluster-network-operator/bindata/network/ovn-kubernetes/ovnkube-node.yaml

Line 169 in 2a56600

mountPropagation: HostToContainer

yes, good points @danwinship I updated the description and am spinning up a bug now

btw, @dougbtv, if we merge this, do we need #645 in 4.4 as well?

openshift-ci-robot · 2020-06-16T19:06:37Z

@haircommander: This pull request references Bugzilla bug 1847675, which is invalid:

expected the bug to target the "4.4.z" release, but it targets "---" instead
expected dependent Bugzilla bug 1847674 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is NEW instead
expected dependent Bugzilla bug 1847674 to target a release in 4.5.0, 4.5.z, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Bug 1847675: [4.4] mount /var/run/netns rslave in ovnkube

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

haircommander · 2020-06-16T19:06:39Z

/bugzilla refresh

openshift-ci-robot · 2020-06-16T19:06:41Z

@haircommander: This pull request references Bugzilla bug 1847675, which is invalid:

expected the bug to target the "4.4.z" release, but it targets "---" instead
expected dependent Bugzilla bug 1847674 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is NEW instead
expected dependent Bugzilla bug 1847674 to target a release in 4.5.0, 4.5.z, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

haircommander · 2020-06-16T19:09:49Z

/bugzilla refresh

openshift-ci-robot · 2020-06-16T19:09:51Z

@haircommander: This pull request references Bugzilla bug 1847675, which is invalid:

expected dependent Bugzilla bug 1847674 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is MODIFIED instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

haircommander · 2020-06-17T18:58:57Z

/bugzilla refresh

openshift-ci-robot · 2020-06-17T18:59:04Z

@haircommander: This pull request references Bugzilla bug 1847675, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.4.z) matches configured target release for branch (4.4.z)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 1847674 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
dependent Bugzilla bug 1847674 targets the "4.5.0" release, which is one of the valid target releases: 4.5.0, 4.5.z
bug has dependents

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

danwinship · 2020-06-22T14:26:50Z

/lgtm

openshift-ci-robot · 2020-06-22T14:27:08Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danwinship, haircommander

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [danwinship]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2020-06-24T19:28:17Z

@haircommander: All pull requests linked via external trackers have merged: openshift/cluster-network-operator#669. Bugzilla bug 1847675 has been moved to the MODIFIED state.

Details

In response to this:

Bug 1847675: [4.4] mount /var/run/netns rslave in ovnkube

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot requested review from pecameron and rcarrillocruz June 15, 2020 16:53

pecameron reviewed Jun 15, 2020

View reviewed changes

haircommander changed the title ~~mount /var/run/netns rslave in ovnkube~~ [4.4] mount /var/run/netns rslave in ovnkube Jun 16, 2020

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 16, 2020

haircommander changed the title ~~[4.4] mount /var/run/netns rslave in ovnkube~~ Bug 1847675: [4.4] mount /var/run/netns rslave in ovnkube Jun 16, 2020

openshift-ci-robot added the bugzilla/severity-unspecified Referenced Bugzilla bug's severity is unspecified for the PR. label Jun 16, 2020

openshift-ci-robot added the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Jun 16, 2020

openshift-ci-robot assigned danwinship Jun 22, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 22, 2020

shawn-hurley added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Jun 24, 2020

openshift-merge-robot merged commit 96a6338 into openshift:release-4.4 Jun 24, 2020

haircommander mentioned this pull request Jun 26, 2020

[release-4.4] mount /var/run/netns rslave in ovnkube #600

Closed

haircommander mentioned this pull request Feb 23, 2022

crio: mount /run rslave openshift/multus-cni#120

Merged

Conversation

haircommander commented Jun 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Jun 15, 2020

Uh oh!

haircommander commented Jun 15, 2020

Uh oh!

haircommander commented Jun 15, 2020

Uh oh!

haircommander commented Jun 15, 2020

Uh oh!

pecameron left a comment

Choose a reason for hiding this comment

Uh oh!

danwinship commented Jun 16, 2020

Uh oh!

danwinship commented Jun 16, 2020

Uh oh!

openshift-ci-robot commented Jun 16, 2020

Uh oh!

openshift-ci-robot commented Jun 16, 2020

Uh oh!

haircommander commented Jun 16, 2020

Uh oh!

openshift-ci-robot commented Jun 16, 2020

Uh oh!

haircommander commented Jun 16, 2020

Uh oh!

openshift-ci-robot commented Jun 16, 2020

Uh oh!

haircommander commented Jun 16, 2020

Uh oh!

openshift-ci-robot commented Jun 16, 2020

Uh oh!

haircommander commented Jun 17, 2020

Uh oh!

openshift-ci-robot commented Jun 17, 2020

Uh oh!

danwinship commented Jun 22, 2020

Uh oh!

openshift-ci-robot commented Jun 22, 2020

Uh oh!

openshift-ci-robot commented Jun 24, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

haircommander commented Jun 15, 2020 •

edited

Loading