Bug 1888565: daemon: Explicitly start rpm-ostreed by cgwalters · Pull Request #2291 · openshift/machine-config-operator

cgwalters · 2020-12-09T21:21:46Z

See https://bugzilla.redhat.com/show_bug.cgi?id=1888565
for one instance of this. Since the DBus activation timeout is 25s
but the systemd timeout is 90s, let's defer to the latter by explicitly
starting the service. (This is slightly racy because rpm-ostreed
auto-exits on idle, but when we go to send the DBus message we'll
auto-activate again if really necessary).

Also tell systemd we're going to need the service via the
firstboot unit file.

See https://bugzilla.redhat.com/show_bug.cgi?id=1888565 for one instance of this. Since the DBus activation timeout is 25s but the systemd timeout is 90s, let's defer to the latter by explicitly starting the service. (This is slightly racy because rpm-ostreed auto-exits on idle, but when we go to send the DBus message we'll auto-activate again if really necessary). Also tell systemd we're going to need the service via the firstboot unit file.

openshift-ci-robot · 2020-12-09T21:21:52Z

@cgwalters: This pull request references Bugzilla bug 1888565, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.7.0) matches configured target release for branch (4.7.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Details

In response to this:

Bug 1888565: daemon: Explicitly start rpm-ostreed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ashcrow · 2020-12-10T00:30:16Z

/retest

sinnykumari · 2020-12-10T13:48:36Z

Few observation:

We are explicitly starting rpm-ostreed daemon only before fetching OSImageURL. Won't we have to ensure same at other places where we perform rpm-ostree operations?
This seems like a workaround to me. For long term solution, are we planning to make changes in rpm-ostree? Perhaps when user calls rpm-ostree functionality, rpm-ostree first checks and starts rpm-ostreed if not already running.

cgwalters · 2020-12-10T15:07:29Z

We are explicitly starting rpm-ostreed daemon only before fetching OSImageURL. Won't we have to ensure same at other places where we perform rpm-ostree operations?

I think the main problem is during OS boot, there's a lot going on, a lot of services starting at once competing for I/O on the disk.

This seems like a workaround to me. For long term solution, are we planning to make changes in rpm-ostree? Perhaps when user calls rpm-ostree functionality, rpm-ostree first checks and starts rpm-ostreed if not already running.

That already happens today - that's how it's all working without this! The problem as noted in the commit message is the disparity between the DBus activation timeout and the systemd one.

But yes, we will likely work around this in rpm-ostree upstream too. It'll just take much longer to land. (And in any case I think the last bit of Wants= is better regardless because it tells systemd explicitly what's happening)

sinnykumari · 2020-12-10T15:59:37Z

Thanks for the explanation, makes sense then.

sinnykumari · 2020-12-10T16:07:26Z


+// Start ensures the daemon is running; the DBus activation timeout
+// is shorter than the systemd timeout.  xref
+// https://bugzilla.redhat.com/show_bug.cgi?id=1888565#c3


Just wanted to confirm, this solves referenced bug but it doesn't solve bug https://bugzilla.redhat.com/show_bug.cgi?id=1888565 where we are seeing rpm-ostree cleanup -r failing because of slow disk, right?

sinnykumari · 2020-12-10T18:18:50Z

/lgtm

openshift-ci-robot · 2020-12-10T18:19:10Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, sinnykumari

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [cgwalters,sinnykumari]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2020-12-10T18:27:47Z

/retest