The /var/run -> /run symlink is missing in the base OS container image

[ggiguash]

The /var/run and /run paths are often used interchangeably. If /var/run symlink does not exist when an OS is booted, a new directory is created instead and we end up in a situation when /var/run and /run are not in sync.

The workaround to this issue is very simple. The following needs to be added to a Containerfile:

RUN ln -s /run /var/

However, different contents in /var/run and /run may lead to problems that are not easy to predict or troubleshoot. Therefore, I'd like to suggest adding /var/run -> /run symlink to the OS container base image.

An example: DBus and NetworkManager Services Cannot Communicate

DBus service creates its socket in /run/dbus directory, but NetworkManager tries to access this socket using /var/run/dbus location. This leads to failure of NetworkManager service with cryptic messages.

Error: Could not create NMClient object: Could not connect: No such file or directory.

The error hints about DBus problems, but the DBus service is up and running. One of the ways to troubleshoot this issue is to trace nmcli command system calls to understand how it tries to communicate with the DBus service.

[cgwalters]

the OS container base image.

There isn't "the" image here, this is the responsibility of the people producing them. This project is independent of them, in the same way podman is.

The following needs to be added to a Containerfile:

Yes, that's OK and valid. However, this is also a bug to fix in the OS components/packages which still reference /var/run as a directory and try to create things there at install time. We filed one recently for postgres https://src.fedoraproject.org/rpms/postgresql16/pull-request/5 for example. (But that's just one package in one distribution, others may need similar fixes)

Hmm, a lint for this would be great to have actually, cc #381

But if you're affected by this the simple thing to do e.g.:

RUN apt|dnf|whatever install -y NetworkManager && rm /var/run -rf

Because the container image should not have this content to start with. The symlink should be created on firstboot via systemd tmpfiles.d.

[cgwalters]

I guess in theory, we could change bootc to fatally error out when trying to deploy an image in this setup. Yeah...the more I think about it the more I like the idea of running the lints in error mode at deploy time. We should support doing that in a symmetric way. Let's use #381 to track that.

(We could just mask the problem and always blow away the files that are there, but I am not a huge fan of stuff like this as it could quickly escalate).

But for now again please contact the people maintaining the OS/distribution base image and the packages for real fixes as I don't think we can in a maintainable/scalable way be papering over those issues here.

[ggiguash]

@cgwalters, let me follow-up with the base image maintainers.
Let's close this issue?