Failure to build packages with /tmp mounted with "nodev" option (recent distros)

[ahartmetz]

I am running e2factory on Kubuntu 24.10, which mounts /tmp with "nosuid,nodev" options among others. I ran into some very strange build failures that I traced to these lines from a build script:

if ! sed --help 2>/dev/null 1>&2 || ! pwd  2>/dev/null 1>&2; then
        echo 'Postinstall script of toolchain-packages.binutils-host-dev package failed!'
        echo 'This script is needed to fix path entries of libtool files.'
        echo 'The required tools "sed" and "pwd" are not available!'
        exit 1
fi

and the non-usability of /dev in the build root filesystem under /tmp. The script prints the error message and aborts as if sed or pwd were not available.
Several workarounds, like setting E2_TMPDIR, are possible, but it should also work out of the box.

[hartwork]

I cannot confirm nodev to take part in the problem…

# mount | grep " /tmp"
TMPFS on /tmp type tmpfs (rw,nodev,relatime,size=2097152k)

# if ! sed --help 2>/dev/null 1>&2 || ! pwd  2>/dev/null 1>&2; then echo TROUBLE ; fi
<no output>

…so far and there is nothing in the call to sed and pwd that would ask for need to create temporary files.

[ahartmetz]

Hm. Here is the complete "mount" output for /tmp in Kubuntu 24.10:
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,nr_inodes=1048576,inode64)
Could it be the nosuid? Does e2factory use the global /tmp or the user's by default? For my user, it's:
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=3206132k,nr_inodes=801533,mode=700,uid=1000,gid=1000,inode64)

Note that env | grep TMP and env | grep TEMP both return nothing, so I think I'm not overriding the temp dir unless I set E2_TMPDIR.

[hartwork]

@ahartmetz hello! I'm afraid I cannot answer that easily (and I should note that I'm only a visitor here). I think your best chance for a fix is probably a minimal reproducer, just an idea.

[ahartmetz]

OK, I will experiment with mount flags, I guess.

[DerDakon]

It is "nodev". It's the usage of /dev/null here that creates the problem. There is simply no way to build anything with nodev as /dev/null is used by every configure script and whatnot.

[hartwork]

@DerDakon are we talking about absolute /dev/null or relative ${TEMPDIR}/dev/null? The latter, correct? For the former I would not see how nodev tmpfs would effect that path. Would love to understand.

[DerDakon]

Everything you see inside an e2factory project, i.e. scripts in proj/init as well as every build-script, is executed inside the chroot environment created by e2-build. And this chroot is created in your directory set by E2TMPDIR, and if that does not exist TMPDIR, i.e. /tmp. So if you have not set any of those to a location mounted without nodev it will end up in /tmp and be affected by this. You could direct that somewhere into your home directory, but I would strongly suggest to really do it on a tmpfs for performance reasons.

[hartwork]

@DerDakon I'm with you that tmpfs would be recommend. In my tests nodev does not prevent creation of ${TEMPDIR}/dev/null but makes the created device unusable (which would explain what @ahartmetz is witnessing). For proof:

# sudo mount -o nodev,remount /tmp/

# mount | grep " /tmp"
TMPFS on /tmp type tmpfs (rw,nodev,relatime,size=2097152k)
                             ^^^^^
# sudo mknod -m 666 /tmp/dev_null c 1 3

# ls -l /tmp/dev_null
crw-rw-rw- 1 root root 1, 3 Mär 17 15:28 /tmp/dev_null

# echo ffffffffff > /tmp/dev_null
bash: /tmp/dev_null: Permission denied  # i.e. unusable

# sudo rm /tmp/dev_null

# sudo mount -o dev,remount /tmp/
                ^^^
# mount | grep " /tmp"
TMPFS on /tmp type tmpfs (rw,relatime,size=2097152k)

# sudo mknod -m 666 /tmp/dev_null c 1 3

# ls -l /tmp/dev_null
crw-rw-rw- 1 root root 1, 3 Mär 17 15:30 /tmp/dev_null  # same as above

# echo ffffffffff > /tmp/dev_null
<no error output>  # i.e. works

So what e2factory could potentially do (and does not currently do?) is to probe the temporary directory like that^^ and error out early with a to-the-point error message saying that the tempdir provided will not allow creation of working devices (if the user running e2factory has enough permissions to mknod).

[ahartmetz]

I also discussed this with someone at the emlix booth at Embedded World. There, we also concluded that a precise warning or error message would save a lot of time. It would have taken me a minute instead of two hours when I first ran into this problem (especially if the message mentioned E2_TMPDIR).
By the way, I notice little to no difference in my home directory. It's an NVMe drive (I suspect any decent SSD would do), common Linux filesystems perform great, and (write) caching is extremely effective for compilation. I still have some mechanical drives and have some build dirs on these, too - no problem, it's all cached.