selinux cloud-init-hotplugd.socket not having permissions to fifo sockets

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1936229

Launchpad details

affected_projects = ['centos']
assignee = None
assignee_name = None
date_closed = None
date_created = 2021-07-14T15:09:37.661517+00:00
date_fix_committed = None
date_fix_released = None
id = 1936229
importance = medium
is_complete = False
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1936229
milestone = None
owner = chad.smith
owner_name = Chad Smith
private = False
status = triaged
submitter = chad.smith
submitter_name = Chad Smith
tags = []
duplicates = []

Launchpad user Chad Smith(chad.smith) wrote on 2021-07-14T15:09:37.661517+00:00

Initial cloud-init-hotplugd.socket has some undesirable interactions with an enforcing SElinux system when trying to listen on a FIFO socket.

Deploying on rocky linux 8.4 we can see SELinux errors preventing the cloud-init-hotplugd.socket from starting

from journalctl -b 0:

systemd[1]: cloud-init-hotplugd.socket: Failed to listen on sockets: Permission denied
systemd[1]: cloud-init-hotplugd.socket: Failed with result 'resources'.
systemd[1]: Failed to listen on cloud-init hotplug hook socket.

...

setroubleshoot[772]: SELinux is preventing systemd from add_name access on the directory hook-hotplug-cmd. For complete SELinux messages run: sealert -l 8969a264-6637-489a-a329-0aafc0b8ee3a
setroubleshoot[772]: SELinux is preventing systemd from add_name access on the directory hook-hotplug-cmd.

                                                                            *****  Plugin catchall (100. confidence) suggests   **************************
                                                                            
                                                                            If you believe that systemd should be allowed add_name access on the hook-hotplug-cmd directory by default.
                                                                            Then you should report this as a bug.
                                                                            You can generate a local policy module to allow this access.
                                                                            Do
                                                                            allow this access for now by executing:
                                                                            # ausearch -c 'systemd' --raw | audit2allow -M my-systemd
                                                                            # semodule -X 300 -i my-systemd.pp

[rocky@ip-172-31-3-239 ~]$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33

[rocky@ip-172-31-3-239 ~]$ systemctl status cloud-init-hotplugd.socket
● cloud-init-hotplugd.socket - cloud-init hotplug hook socket
Loaded: loaded (/usr/lib/systemd/system/cloud-init-hotplugd.socket; enabled; vendor preset: disabled)
Active: failed (Result: resources)
Listen: /run/cloud-init/hook-hotplug-cmd (FIFO)

Jul 14 03:39:49 ip-172-31-3-239.us-east-2.compute.internal systemd[1]: cloud-init-hotplugd.socket: Failed to listen on sockets: Permission denied
Jul 14 03:39:49 ip-172-31-3-239.us-east-2.compute.internal systemd[1]: cloud-init-hotplugd.socket: Failed with result 'resources'.
Jul 14 03:39:49 ip-172-31-3-239.us-east-2.compute.internal systemd[1]: Failed to listen on cloud-init hotplug hook socket.

[rocky@ip-172-31-3-239 ~]$ systemctl status cloud-init-hotplugd.service
● cloud-init-hotplugd.service - cloud-init hotplug hook daemon
Loaded: loaded (/usr/lib/systemd/system/cloud-init-hotplugd.service; static; vendor preset: disabled)
Active: inactive (dead)

When setting selinux to permissive, we can see no errors from the systemd services

[rocky@ip-172-31-3-239 ~]$ sudo sed -i 's/SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config
[rocky@ip-172-31-3-239 ~]$ sudo cloud-init clean --logs --reboot

[rocky@ip-172-31-3-239 ~]$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: permissive
Mode from config file: permissive
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33
[rocky@ip-172-31-3-239 ~]$ systemctl status cloud-init-hotplugd.socket
● cloud-init-hotplugd.socket - cloud-init hotplug hook socket
Loaded: loaded (/usr/lib/systemd/system/cloud-init-hotplugd.socket; enabled; vendor preset: disabled)
Active: active (listening) since Wed 2021-07-14 03:53:19 UTC; 1min 16s ago
Listen: /run/cloud-init/hook-hotplug-cmd (FIFO)
Tasks: 0 (limit: 4797)
Memory: 0B
CGroup: /system.slice/cloud-init-hotplugd.socket

Jul 14 03:53:19 ip-172-31-3-239.us-east-2.compute.internal systemd[1]: Listening on cloud-init hotplug hook socket.

Attempting to generate a selinux policy for this systemd.socket I get the following:
[rocky@ip-172-31-3-239 ~]$ sudo ausearch -c 'systemd' --raw | audit2allow -m cloud-init-hotplug

module cloud-init-hotplug 1.0;

require {
type init_t;
type net_conf_t;
class dir add_name;
class fifo_file { create open read write };
}

#============= init_t ==============
allow init_t net_conf_t:dir add_name;
allow init_t net_conf_t:fifo_file { create open read write };

[rocky@ip-172-31-3-239 ~]$ sudo ausearch -c 'systemd' --raw | audit2allow -M cloud-init-hotplug
[rocky@ip-172-31-3-239 ~]$ sudo semodule -i cloud-init-hotplug.pp
[rocky@ip-172-31-3-239 ~]$ sudo sed -i 's/SELINUX=permissive/SELINUX=enforcing/' /etc/selinux/config

[rocky@ip-172-31-3-239 ~]$ sudo cloud-init clean --logs --reboot

[rocky@ip-172-31-3-239 ~]$ sudo sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33
[rocky@ip-172-31-3-239 ~]$
[rocky@ip-172-31-3-239 ~]$
[rocky@ip-172-31-3-239 ~]$
[rocky@ip-172-31-3-239 ~]$ systemctl status cloud-init-hotplugd.socket
● cloud-init-hotplugd.socket - cloud-init hotplug hook socket
Loaded: loaded (/usr/lib/systemd/system/cloud-init-hotplugd.socket; enabled; vendor preset: disabled)
Active: active (listening) since Wed 2021-07-14 03:59:55 UTC; 1min 51s ago
Listen: /run/cloud-init/hook-hotplug-cmd (FIFO)
Tasks: 0 (limit: 4797)
Memory: 0B
CGroup: /system.slice/cloud-init-hotplugd.socket

[ubuntu-server-builder]

Launchpad user Chad Smith(chad.smith) wrote on 2021-07-14T16:45:08.267065+00:00

Here are the proposed systemd socket and service files:

/systemd/cloud-init-hotplugd.socket

[Unit]
Description=cloud-init hotplug hook socket

[Socket]
ListenFIFO=/run/cloud-init/hook-hotplug-cmd

[Install]
WantedBy=cloud-init.target

systemd/cloud-init-hotplugd.service

[Unit]
Description=cloud-init hotplug hook daemon
After=cloud-init-hotplugd.socket

[Service]
Type=simple
ExecStart=/bin/bash -c 'read args <&3; echo "args=$args";
exec /usr/bin/cloud-init devel hotplug-hook $args;
exit 0'
SyslogIdentifier=cloud-init-hotplugd
TimeoutStopSec=5

[ubuntu-server-builder]

Launchpad user yangzhuangzhuang(steven-yang97) wrote on 2022-03-08T13:08:11.103167+00:00

Will the problem be fixed? I think the corresponding SELinux policy should be released for /run/cloud-init/hook-hotplug-cmd

[ubuntu-server-builder]

Launchpad user yangzhuangzhuang(steven-yang97) wrote on 2022-03-08T13:19:14.399312+00:00

And I think if the service depends on socket, it should change After=cloud-init-hotplugd.socket to Requires=cloud-init-hotplugd.socket in the cloud-init-hotplugd.service file.What about you?

[ubuntu-server-builder]

Launchpad user Brett Holman(holmanb) wrote on 2022-03-09T18:49:09.648953+00:00

yangzhuangzhuang: Thanks for reaching out. I believe this problem will be fixed in the future, but we haven't yet started resolving this issue. Since hotplug is new-ish feature (21.4 I believe), I suspect we don't have many users yet. Relative pain caused by this bug is believed to be low currently, as it should only affect users attempting to use both Selinux and hotplug.

Is this issue something that negatively affects your use of cloud-init?

The suggestion to replace "After" with "Requires" seems correct, but I haven't taken the time to verify. If you would like to see this bug resolved faster PRs (on Github) are always welcome.

[ubuntu-server-builder]

Launchpad user yangzhuangzhuang(steven-yang97) wrote on 2022-03-14T12:05:59.865217+00:00

#1335

[ubuntu-server-builder]

Launchpad user shixuantong(sxt1001) wrote on 2022-12-02T09:12:02.837819+00:00

Hello, this problem has been reported for over a year, is there any plan to fix it now?

[ubuntu-server-builder]

Launchpad user James Falcon(falcojr) wrote on 2022-12-02T16:03:29.015702+00:00

This is a bug that should be fixed, but there are no immediate plans to fix it in the near future unless we see a strong demand causing it to be higher priority.

[ubuntu-server-builder]

Launchpad user shixuantong(sxt1001) wrote on 2022-12-09T03:20:37.507530+00:00

I find /run/cloud-init/hook-hotplug-cmd is a pipeline file. Why does it say it's a directory?

[ubuntu-server-builder]

Launchpad user shixuantong(sxt1001) wrote on 2022-12-09T04:09:01.602645+00:00

if we generate a selinux policy for this systemd.socket, what are the impacts or risks of successfully starting cloud-init-hotplugd?

[ubuntu-server-builder]

Launchpad user Brett Holman(holmanb) wrote on 2022-12-12T13:36:20.360014+00:00

I find /run/cloud-init/hook-hotplug-cmd is a pipeline file. Why does it say it's a directory?

I'm not sure what you are referring to. More details, please.

if we generate a selinux policy for this systemd.socket, what are the impacts or risks of successfully starting cloud-init-hotplugd?

Per the docs[1]: "When the hotplug event is supported by the data source and configured in user data, cloud-init will respond to the addition or removal of network interfaces to the system."

For a user that has not configured this in user-data, risk should be low.

I would say that the risks of fixing this bug (to successfully start cloud-init-hotplugd on selinux systems) are therefore minimal.

[1] https://cloudinit.readthedocs.io/en/latest/topics/events.html#hotplug

[holmanb]

This is a downstream issue, so I'm closing it. I've filed this issue against the affected downstream.