[rfc] native aarch64 builders

[dkwo]

Cross-compilation is subtle.

1. any mainteiners interested? @leahneukirchen @classabbyamp @paper42
2. any users interested in supporting this?
3. hardware suggestions

I'd be happy to donate some money to purchase hardware,
if you think it's doable and enough people are interested.

[Eloitor]

I'm interested in this as a user. I would like to use void in my pinephone. I tried to create a template for osmin (an OSM app) but it failed to build for aarch64.

[classabbyamp]

you can already compile natively for any arch yourself

[dkwo]

Compiling natively is fine, but you'd have to rebuild lots of subtly miscompiled stuff first, like qmake, libtool, etc, and build stuff like sagemath yourself, while having aarch64 as a first-class citizen (i.e. native) would avoid all of this.

[Eloitor]

you can already compile natively for any arch yourself

Compiling in the pinephone is a nightmare

[leahneukirchen]

I think the main problem is getting adequate hardware as an aarch64 builder, as server-grade aarch64 systems are still quite expensive; and most of them cannot run armv7 code, so it can't be a builder for both archs.

Also this merely shifts the burden of fixing cross-compiling to the other architectures.

[tornaria]

I have a rpi 4 with 8 GB ram and I will start testing some math software on it. It would be nice to have good support, we have some key packages that won't cross compile (ntl). But if some cross-compiled packages in the build chain are broken, that complicates matters a lot.

It seems aarch64 is more important than i686 at this point (e.g. a few days ago I learned that sagemath is completely broken on i686, but nobody noticed -- still we spend a lot of energy fixing 32 bit issues and 8087 magic in particular since we can't even assume sse2)

If it's not possible to get aarch64 builders, is there a chance that some (few, key, carefully selected) packages can be natively compiled on a x86_64 builder using qemu instead of cross?

[leahneukirchen]

An rpi 4 is not fast enough to keep up as a full builder.

[dkwo]

But if some cross-compiled packages in the build chain are broken, that complicates matters a lot.

Exactly, see libtool, possibly qmake.

[dkwo]

@leahneukirchen How about an M1 Mac mini from Apple?

[leahneukirchen]

Doesn't run armv7 code either. Probably fast enough, may need external storage.

Not sure if the different page size is problematic.

[dkwo]

Page size affects a few packages (I assume that https://github.com/AsahiLinux/docs/wiki/Broken-Software also applies in the reverse way), among them zig; many have already been fixed.
@leahneukirchen Would you be interested in receiving something like that as a gift, provided enough people contribute?

[paper42]

It is simple to modify xbps-src to use qemu, just an example:

--- a/common/xbps-src/shutils/chroot.sh
+++ b/common/xbps-src/shutils/chroot.sh
@@ -160,7 +160,7 @@ chroot_handler() {
 
     if [ "$action" = "chroot" ]; then
         $XBPS_COMMONDIR/chroot-style/${XBPS_CHROOT_CMD:=uunshare}.sh \
-            $XBPS_MASTERDIR $XBPS_DISTDIR "$XBPS_HOSTDIR" "$XBPS_CHROOT_CMD_ARGS" /bin/xbps-shell
+            $XBPS_MASTERDIR $XBPS_DISTDIR "$XBPS_HOSTDIR" "$XBPS_CHROOT_CMD_ARGS" /usr/bin/qemu-aarch64-static /bin/bash /bin/xbps-shell
         rv=$?
     else
         env -i -- PATH="/usr/bin:$PATH" SHELL=/bin/sh \
@@ -178,7 +178,7 @@ chroot_handler() {
             ${XBPS_ALT_REPOSITORY:+XBPS_ALT_REPOSITORY=$XBPS_ALT_REPOSITORY} \
             $XBPS_COMMONDIR/chroot-style/${XBPS_CHROOT_CMD:=uunshare}.sh \
             $XBPS_MASTERDIR $XBPS_DISTDIR "$XBPS_HOSTDIR" "$XBPS_CHROOT_CMD_ARGS" \
-            /void-packages/xbps-src $XBPS_OPTIONS $action $pkg
+            /usr/bin/qemu-aarch64-static /bin/bash /void-packages/xbps-src $XBPS_OPTIONS $action $pkg
         rv=$?
     fi
 

echo XBPS_CHROOT_CMD=bwrap >> etc/conf
./xbps-src binary-bootstrap -m maarch aarch64-musl
cp /usr/bin/qemu-aarch64-static maarch/usr/bin/
./xbps-src -m maarch pkg libtool
./xbps-src -m maarch pkg ntl

This assumes bubblewrap, qemu-user-static and binfmt are available on the host. It would have to be improved if it was meant to be merged to void-packages and it's very slow.

[leahneukirchen]

Would you be interested in receiving something like that as a gift, provided enough people contribute?

I don't have resources to run this myself locally atm.

[vincele]

RK3588 support is coming real soon now (tm). This will be faster than rpi4, and less expensive than apple HW.