musl build for cross-platform docker images

[joepio]

Musl images can be even lighter, as it includes used libraries we can skip a runtime like alpine. It should also eliminate some runtime issues I've had in the past (related to glibc mostly).

But I think we can't build musl for aarch64 because of a problem in ring. So I either have to remove multi-platform docker builds, or wait for an upstream fix.

What I tried

See musl-docker branch

Example of failed CI action

#18 1218.1   cc1: error: unrecognized command-line option '-m64'

• Tried adding ENV RUSTFLAGS='-C linker=x86_64-linux-gnu-gcc'
• Tried adding linux-musl-dev dependency
• Tried clux/muslrust/
• Tried even more envs, no luck

I think the problem is that Cargo is using the wrong linker while building on ARM64.

I also think this is wrong: --target x86_64-unknown-linux-musl as I should probably have aarch64-unknown-linux-musl. But how do I differentiate the target between builds? I use this docker push action which deals with pushing various builds. But I can't pass opts here.

      - name: Build and push Docker image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          provenance: false
          platforms: linux/amd64,linux/arm64
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

Should I get rid of this action and manually use matrix?

Or is there a way I can use a --target that is dynamic / conditional on the current architecture?

[AlexMikhalev]

Musl docker file: https://github.com/applied-knowledge-systems/atomic-data-rust/blob/master/Dockerfile_musl
Also, this Earthfile can be adapted to multiplatform, see line. I can create PR if you are ok with it and set Earthly based github action. Test pipeline can be run locally with earthly -i test-pipeline.

[joepio]

I tried doing a musl build again, but it looks like I still need libgcc_s... It's weird.

/ # ./atomic-server-bin --wda
/bin/ash: ./atomic-server-bin: not found
/ # ldd /atomic-server-bin
	/lib64/ld-linux-x86-64.so.2 (0x7f7055836000)
Error loading shared library libgcc_s.so.1: No such file or directory (needed by /atomic-server-bin)
	librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7f7055836000)
	libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f7055836000)
	libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f7055836000)
	libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7f7055836000)
	libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f7055836000)
Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /atomic-server-bin)

EDIT: after running apk add libgcc libc6-compat, I get Error relocating /atomic-server-bin: __res_init: symbol not found when running /atomic-server-bin.

[joepio]

Just checked again, still cant compile musl on my M1 mac:

error: failed to run custom build command for `ring v0.16.20`

Caused by:
  process didn't exit successfully: `/Users/joep/dev/github/atomicdata-dev/atomic-server/target/debug/build/ring-d10ff396161ecc90/build-script-build` (exit status: 101)
  --- stdout
  OPT_LEVEL = Some("0")
  TARGET = Some("x86_64-unknown-linux-musl")
  HOST = Some("aarch64-apple-darwin")
  CC_x86_64-unknown-linux-musl = None
  CC_x86_64_unknown_linux_musl = None
  TARGET_CC = None
  CC = None
  CROSS_COMPILE = None
  CFLAGS_x86_64-unknown-linux-musl = None
  CFLAGS_x86_64_unknown_linux_musl = None
  TARGET_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")

  --- stderr
  running "musl-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-U_FORTIFY_SOURCE" "-DNDEBUG" "-c" "-o/Users/joep/dev/github/atomicdata-dev/atomic-server/target/x86_64-unknown-linux-musl/debug/build/ring-5e4aa2f470a7900e/out/aesni-x86_64-elf.o" "/Users/joep/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.16.20/pregenerated/aesni-x86_64-elf.S"
  thread 'main' panicked at /Users/joep/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.16.20/build.rs:653:9:
  failed to execute ["musl-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-U_FORTIFY_SOURCE" "-DNDEBUG" "-c" "-o/Users/joep/dev/github/atomicdata-dev/atomic-server/target/x86_64-unknown-linux-musl/debug/build/ring-5e4aa2f470a7900e/out/aesni-x86_64-elf.o" "/Users/joep/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.16.20/pregenerated/aesni-x86_64-elf.S"]: No such file or directory (os error 2)
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

[joepio]

We musl now!