Usage of `nc` is incompatible with `nmap-ncat` in Fedora/CentOS/RHEL

[Conan-Kudo]

Bug report

cloud-init recently grew a requirement on nc, but the usage of nc relies on a flag that only exists in OpenBSD netcat: -W (Terminate after receiving recvlimit packets from the network).

The ncat(1) man page does not indicate that there is an equivalent option from Nmap's Netcat implementation.

In the Enterprise Linux family, the only netcat implementation that is provided is the one from Nmap. It is provided as nmap-ncat in Fedora/CentOS, and ncat in Debian/Ubuntu.

This incompatibility makes cloud-init fail on Enterprise Linux platforms now.

Environment details

• Cloud-init version: 25.1
• Operating System Distribution: Fedora ELN / CentOS Stream 10 / Red Hat Enterprise Linux 10 / AlmaLinux 10
• Cloud provider, platform or installer type: All

[TheRealFalcon]

@Conan-Kudo , does the patch mentioned here work for you? #5489 (comment)

[sgallagher]

@TheRealFalcon Not necessarily, as we don't actually guarantee that python exists on our smallest media (it's becoming a whole other issue with regards to managing systems with Ansible, but I digress...)

The discussion on #5489 strongly suggests that the switch was made for optimization reasons, so given the choice, it would be best for us to be able to use nmap-ncat rather than switch to Python, as it would be much closer in performance to the openbsd-netcat implementation. (I haven't tested them to see if one outperforms the other, but I'm certain both are faster than Python).

Edit: I didn't realize when I wrote the above that the rest of cloud-init is Python, so I guess that concern about python not existing doesn't really matter here. The performance impact of a much slower implementation of netcat remains, however.

[ani-sinha]

@holmanb @TheRealFalcon could we have a non-python based solution please? Seems a very large number of distros are affected by the dependency change. Looking at your comment when will we have a compiled in nc solution that will resolve this dilemma?

[holmanb]

Hi @ani-sinha, no progress yet on that. What was wrong with the suggested Python patch?

[ani-sinha]

Hi @ani-sinha, no progress yet on that. What was wrong with the suggested Python patch?

It would slow down the boot up process as you have pointed out. Seems it's trivial to have something written in c.

[holmanb]

It would slow down the boot up process as you have pointed out.

True, it might. We might be able to make nmap's ncat work, however.

It looks like "bi-directional" datagram mode isn't supported by nmap's netcat at all. OpenBSD's nc explicitly supports this (which cloud-init uses) (man 1 nc):

       -s sourceaddr
               Set the source address to send packets from, which is  useful  on  machines  with
               multiple  interfaces.  For Unix-domain datagram sockets, specifies the local tem‐
               porary socket file to create and use so that datagrams can be  received.   Cannot
               be used together with -x.

However, I don't think that there is anything preventing us from switching to a stream socket instead for better distro compatibility. For example with a test server:

#!/usr/bin/python3
from socket import AF_UNIX, SOCK_STREAM, SOCK_CLOEXEC, socket
import os

path =  "/tmp/tmp.sock"
sock = socket(AF_UNIX, SOCK_STREAM| SOCK_CLOEXEC)
sock.bind(path)
sock.listen(1)
conn, _ = sock.accept()

# block until a message is received - this may take a long time
chunk, remote = conn.recvfrom(5)
if b"start" != chunk:
    sock.close()
    raise Exception(f"Received invalid message: [{chunk}]")
conn.sendall(b"done")
conn.close()
os.unlink(path)

Both nmap's ncat and openbsd's nc work as a clients to the server:

# pick one
$ echo "start" | nc.openbsd -U /tmp/tmp.sock
$ echo "start" | ncat -U /tmp/tmp.sock > /dev/null

This would require some changes in cloudinit/sockets.py, possibly elsewhere too. If we can verify that it works, I think something like this could be merged into upstream.

fyi @ani-sinha @Conan-Kudo ^^

[ani-sinha]

It would slow down the boot up process as you have pointed out.

True, it might. We might be able to make nmap's ncat work, however.

It looks like "bi-directional" datagram mode isn't supported by nmap's netcat at all. OpenBSD's nc explicitly supports this (which cloud-init uses) (man 1 nc):

       -s sourceaddr
               Set the source address to send packets from, which is  useful  on  machines  with
               multiple  interfaces.  For Unix-domain datagram sockets, specifies the local tem‐
               porary socket file to create and use so that datagrams can be  received.   Cannot
               be used together with -x.

However, I don't think that there is anything preventing us from switching to a stream socket instead for better distro compatibility. For example with a test server:

#!/usr/bin/python3
from socket import AF_UNIX, SOCK_STREAM, SOCK_CLOEXEC, socket
import os

path = "/tmp/tmp.sock"
sock = socket(AF_UNIX, SOCK_STREAM| SOCK_CLOEXEC)
sock.bind(path)
sock.listen(1)
conn, _ = sock.accept()

block until a message is received - this may take a long time

chunk, remote = conn.recvfrom(5)
if b"start" != chunk:
sock.close()
raise Exception(f"Received invalid message: [{chunk}]")
conn.sendall(b"done")
conn.close()
os.unlink(path)
Both nmap's ncat and openbsd's nc work as a clients to the server:

pick one

$ echo "start" | nc.openbsd -U /tmp/tmp.sock
$ echo "start" | ncat -U /tmp/tmp.sock > /dev/null
This would require some changes in cloudinit/sockets.py, possibly elsewhere too. If we can verify that it works, I think something like this could be merged into upstream.

fyi @ani-sinha @Conan-Kudo ^^

Can we have a fix with this? Its easier for us to test downstream, so if this merged upstream we can pick it up as a part of the next rebase and test.