Kernel before 5.6.X (April 2020): superuser rights needed / setsockopt: Operation not permitted

[fcmircea]

Hello,

I'm trying to run BindToInterface in a script as part of a systemd service that runs under its own separate user.

The service file looks like this:

[Unit]
Description=Deluge Bittorrent Client Daemon
Documentation=man:deluged
After=network-online.target mnt-storage.mount
Requires=mnt-storage.mount
BindsTo=mnt-storage.mount

[Service]
Type=simple
UMask=000

ExecStart=/bin/bash /media/bti/deluged.sh

Restart=on-failure

# Time to wait before forcefully stopped.
TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

The script is as follows:

#!/bin/bash

BIND_INTERFACE=eno2 DNS_OVERRIDE_IP=8.8.8.8 BIND_EXCLUDE=127.0.0.1,192.168. LD_PRELOAD=/media/bti/bindToInterface.so /usr/bin/deluged -d -l /var/log/deluge/daemon.log -L warning

when executing:

systemctl status deluged

I get the following output:

bash[503711]: setsockopt: Operation not permitted

Is there a way to run your tool without elevated privileges?

Thanks

[JsBergbau]

setsockopt normally works as normal user. However I know that systemd enforces some restrictions, like certain directories are not writeable.

Please try curl --interface eno2 ifconfig.me from normal command prompt and from systemd unit. First try should succeed. Second try should give the same error message. If so, I suggest asking at stackoverflow or superuser regarding setsockopt and systemd. Please report your results.

[fcmircea]

Done, posted at: https://stackoverflow.com/questions/69994803/setsockopt-for-systemd-unit-error-operation-not-permited

Let's see if there are any answers.

[birdie-github]

I've got the same issue:

BIND_INTERFACE=tun0 DNS_OVERRIDE_IP=8.8.8.8 LD_PRELOAD=/usr/local/lib/x86_64-linux-gnu/bindToInterface.so curl https://ifconfig.co
setsockopt: Operation not permitted
setsockopt: Operation not permitted
setsockopt: Operation not permitted
setsockopt: Operation not permitted
curl: (6) Could not resolve host: ifconfig.co

however

curl --interface tun0 https://ifconfig.co

works perfectly. The problem is I have to run a different application which cannot bind to any interfaces and I don't know what to do.

I've found some interesting info and posted it here: https://unix.stackexchange.com/questions/679881/setsockopt-operation-not-permitted-error-when-trying-to-use-bindtointerface-u

[JsBergbau]

Hi birdie-github. Could it be an Apparmor / SELinux issue? You could try disabling this.

Somewhere I've read, that binding to interface needs CAP_NET_RAW privilige. You can try by sudo setcap cap_net_raw <Path of your executable>. For testing with curl this would be

sudo setcap cap_net_raw $(eval readlink -f `which curl`)

To remove all capabilities after testing you can use setcap -r /path/to/program, for more information about removing capabilities see https://unix.stackexchange.com/questions/303423/unset-setcap-additional-capabilities-on-excutable

[birdie-github]

# setcap cap_net_raw+eip `which curl`

# getcap `which curl`
/usr/bin/curl = cap_net_raw+eip

Now it produces no errors but it does NOT work:

curl https://ifconfig.co
144.AAA.BBB.CCC

curl --interface tun0 https://ifconfig.co
45.XXX.YYY.ZZZ

BIND_INTERFACE=tun0 DNS_OVERRIDE_IP=8.8.8.8 LD_PRELOAD=/usr/local/lib/x86_64-linux-gnu/bindToInterface.so curl https://ifconfig.co
144.AAA.BBB.CCC

And it cannot work:

https://stackoverflow.com/questions/18058426/does-using-linux-capabilities-disable-ld-preload

Like Oliver Matthews answered, LD_PRELOAD is disabled for both setuid binaries, and for binaries having file capabilities, for security reasons.

strace however shows that with setcap setsockopt(3, SOL_SOCKET, SO_BINDTODEVICE, "tun0\0", 5) succeeds when using curl without any overrides (i.e. with --interface tun0).

[JsBergbau]

Thanks for giving the hint, that setting capabilities will not work with preloaded libraries.
Can you try

sudo setcap -r /usr/local/lib/x86_64-linux-gnu/bindToInterface.so
sudo chown root /usr/local/lib/x86_64-linux-gnu/bindToInterface.so
sudo chmod u+s /usr/local/lib/x86_64-linux-gnu/bindToInterface.so

and try again. Since the source code is public and it is quite it is ok from a security point of view.

[birdie-github]

# setcap -r `which curl` 
# setcap -r `which strace` 

# chown root /usr/local/lib/x86_64-linux-gnu/bindToInterface.so
# chmod u+s /usr/local/lib/x86_64-linux-gnu/bindToInterface.so
# ls -l /usr/local/lib/x86_64-linux-gnu/bindToInterface.so
-rwsr-xr-x 1 root root 11848 Dec  2 12:32 /usr/local/lib/x86_64-linux-gnu/bindToInterface.so

$ BIND_INTERFACE=tun0 DNS_OVERRIDE_IP=8.8.8.8 LD_PRELOAD=/usr/local/lib/x86_64-linux-gnu/bindToInterface.so curl https://ifconfig.co
setsockopt: Operation not permitted
setsockopt: Operation not permitted
setsockopt: Operation not permitted
setsockopt: Operation not permitted
curl: (6) Could not resolve host: ifconfig.co

(# - running under root, $ - running under a normal user account).

I see no AppArmor related messages in system logs.

[birdie-github]

The following commands:

# setcap cap_net_raw+eip /usr/local/lib/x86_64-linux-gnu/bindToInterface.so
# setcap cap_net_raw+eip `which curl`

make the errors disappear but the issue remains.

[JsBergbau]

I could now reproduce the error and try to find a solution. In the meantime you don't need to try anything.

[JsBergbau]

@birdie-github You seem to have also a lot of linux experience.

Soruce-Code of curl

#ifdef SO_BINDTODEVICE
      /* I am not sure any other OSs than Linux that provide this feature,
       * and at the least I cannot test. --Ben
       *
       * This feature allows one to tightly bind the local socket to a
       * particular interface.  This will force even requests to other
       * local interfaces to go out the external interface.
       *
       *
       * Only bind to the interface when specified as interface, not just
       * as a hostname or ip address.
       *
       * interface might be a VRF, eg: vrf-blue, which means it cannot be
       * converted to an IP address and would fail Curl_if2ip. Simply try
       * to use it straight away.
       */
      if(setsockopt(sockfd, SOL_SOCKET, SO_BINDTODEVICE,
                    dev, (curl_socklen_t)strlen(dev) + 1) == 0) {
        /* This is typically "errno 1, error: Operation not permitted" if
         * you're not running as root or another suitable privileged
         * user.
         * If it succeeds it means the parameter was a valid interface and
         * not an IP address. Return immediately.
         */
        return CURLE_OK;
      }
#endif

So sourcecode of curl expects this to fail without root priviliges. Despite it works.
bindToInterface uses exact the same API-call setsockopt(sockfd, SOL_SOCKET, SO_BINDTODEVICE, &interface, sizeof(interface)); //Will fail if socket is already bound to another interface

Perhaps you have an idea what is different there.

[birdie-github]

Despite it works.

No, it fails, I've checked it using strace:

strace curl --interface tun0 https://ifconfig.co 2>&1 | grep setsockopt
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPIDLE, [60], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPINTVL, [60], 4) = 0
setsockopt(3, SOL_SOCKET, SO_BINDTODEVICE, "tun0\0", 5) = -1 EPERM (Operation not permitted)

curl binds to the IP address instead.

Maybe folks at stackexchange have something to say: https://unix.stackexchange.com/questions/679881/setsockopt-operation-not-permitted-error-when-trying-to-use-bindtointerface-u

I've no idea either. Not a lot of actual info on the topic and I'm not a good C programmer to read the kernel source code.

This is definitely an issue with Ubuntu. I cannot reproduce it locally with Fedora 35 despite having SeLinux enabled.

[JsBergbau]

Very strange behaviour
You can test this behaviour even with having a second interface like a VPN by using lo interface. Of course curl can't connect, but the socket binding error will occur before that. That info might help on stackoverflow.

First I thought it depends on the kernel version, because on Raspberry PI with Debian Buster and Kernel version 4.x it fails, whereas it works with kernel version 5.x. However on Ubuntu with Kernel version 5.x it also fails.

Debian Buster amd64 Kernel v4.x

uname -r
4.19.0-9-amd64
strace curl --interface lo ifconfig.me 2>&1 | grep sockopt
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPIDLE, [60], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPINTVL, [60], 4) = 0
setsockopt(3, SOL_SOCKET, SO_BINDTODEVICE, "lo\0", 3) = -1 EPERM (Die Operation ist nicht erlaubt)

Debian Buster Kernel 5.x (Raspberry PI)

uname -r
5.10.17-v7+

strace curl --interface lo ifconfig.me 2>&1 | grep sockopt
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPIDLE, [60], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPINTVL, [60], 4) = 0
setsockopt(3, SOL_SOCKET, SO_BINDTODEVICE, "lo\0", 3) = 0

Debian Buster Kernel v4.x (Raspberry PI)

 uname -r
4.19.97+

strace curl --interface lo ifconfig.me 2>&1 | grep sockopt
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPIDLE, [60], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPINTVL, [60], 4) = 0
setsockopt(3, SOL_SOCKET, SO_BINDTODEVICE, "lo\0", 3) = -1 EPERM (Die Operation ist nicht erlaubt)

Ubuntu 20.04.3 LTS

uname -r
5.4.0-91-generic
strace curl --interface lo ifconfig.me 2>&1 | grep sockopt
setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(5, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
setsockopt(5, SOL_TCP, TCP_KEEPIDLE, [60], 4) = 0
setsockopt(5, SOL_TCP, TCP_KEEPINTVL, [60], 4) = 0
setsockopt(5, SOL_SOCKET, SO_BINDTODEVICE, "lo\0", 3) = -1 EPERM (Operation not permitted)

[birdie-github]

Please update the bug title to indicate that it affects Ubuntu/Debian and not necessarily other Linux distros.

And bash[] in its title is absolutely not necessary.

[birdie-github]

Information (courtesy or https://unix.stackexchange.com/users/325065/uncle-billy ):

It seems that the ability to do that as a regular user is pretty new (5.6.X - April 2020). With older kernels you need CAP_NET_RAW.

Google tells me that the kernel in Ubuntu 18.04.6 is 4.15. So you need to either upgrade or setcap cap_net_raw+ep your_program

Have the library pass the socket fd (via scm_rights on a Unix socket) to a setcap executable which sets that option on it. You cannot gain privileges from a library -- the main executable should be setuid or setcap

So, the issue is down to an old kernel.

I wonder if you could code this feature.