CI: fix ARM64 image build error & enable compilation

[Ts-sound]

Overview

Fix ARM64 build failures caused by:

1. QEMU ARM64 random segmentation fault bug in older versions
2. Incorrect armv9.2-a detection in CMake (compiler only supports up to armv8.6-a)

Changes:

• Updated QEMU to tonistiigi/binfmt:qemu-v10.2.1
• Temporarily disabled armv9.2 detection in CMake (requires further review by maintainers)

Additional Information

Related Issues:

• No linux/arm64 images are available #19177
• [Tracker] Docker build fails on CI for arm64 #11888

Verified Build: https://github.com/Ts-sound/llama.cpp/actions/runs/23467958136

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES
  • AI assisted in summarizing my test notes

[ehfd]

I affirm that the below (using GCC 14) works well for cuda-new.Dockerfile. I believe that aarch64 builds for CUDA can be enabled as well, and it will help a lot.

build-and-push-job:
  stage: build-and-push
  variables:
    LLAMA_CPP_REF: "b8477"
  script:
  - cd $CI_PROJECT_DIR
  - git clone https://github.com/ggml-org/llama.cpp.git
  - cd llama.cpp
  - git checkout $LLAMA_CPP_REF
  - |
  - sed -i 's/apt-get install -y build-essential/apt-get install -y gcc-14 g++-14 build-essential/g' .devops/cuda-new.Dockerfile
  - sed -i '/apt-get install -y gcc-14 g++-14 build-essential/a ENV CC=gcc-14 CXX=g++-14 CUDAHOSTCXX=g++-14' .devops/cuda-new.Dockerfile
  - |
    # Build cuda-new.Dockerfile
    docker buildx build \
      -f .devops/cuda-new.Dockerfile \
      --push \
      --provenance=false \
      --platform linux/arm64 \
      --target full \
      -t $CI_REGISTRY_IMAGE:latest \
      -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA \
      .

[taronaeo]

@ggerganov Can we get some assistance to trigger the CI? :)

[ehfd]

@Ts-sound Since you just reverted your arm9.2 + SME comment [CI: revert ggml/src/ggml-cpu/CMakeLists.txt](https://github.com/ggml-org/llama.cpp/pull/20929/changes/7440e21bddd26347acecbf67fc8fbc4db1fbf76a); you need GCC 14 in Ubuntu 24.04 for the compiling to work.

[Ts-sound]

SME features require GCC 14 + , but upgrading the compiler is not up to me to decide. @ehfd

[ggerganov]

@ggerganov Can we get some assistance to trigger the CI? :)

Do you still need help - I think you should be able to trigger now?

[CISC]

Any reason not to set runs_on to 24.04 on all (even s390x)?

[taronaeo]

Any reason not to set runs_on to 24.04 on all (even s390x)?

Ah, it was initially to test if armv9 is recognised with ubuntu 24.04 / GCC 13. Let's shift all to 24.04 now :)

[ehfd]

This is the error I face while building the CUDA image in ARM for GH200 or DGX Spark. The same issue exists in the CPU build.

#12 198.5 cc1: error: invalid feature modifier 'sme' in '-march=armv9.2-a+dotprod+fp16+sve+i8mm+sve2+sme'
#12 198.5 cc1: note: valid arguments are: fp simd crc lse fp16 rcpc rdma dotprod aes sha2 crypto sha3 sm4 fp16fml sve profile rng memtag sb ssbs predres sve2 sve2-sm4 sve2-aes sve2-sha3 sve2-bitperm tme i8mm f32mm f64mm bf16 flagm pauth ls64 mops cssc; did you mean 'sm4'?
#12 198.5 cc1: error: invalid feature modifier 'sme' in '-march=armv9.2-a+dotprod+fp16+sve+i8mm+sme'
#12 198.5 cc1: note: valid arguments are: fp simd crc lse fp16 rcpc rdma dotprod aes sha2 crypto sha3 sm4 fp16fml sve profile rng memtag sb ssbs predres sve2 sve2-sm4 sve2-aes sve2-sha3 sve2-bitperm tme i8mm f32mm f64mm bf16 flagm pauth ls64 mops cssc; did you mean 'sm4'?
#12 198.6 gmake[2]: *** [ggml/src/CMakeFiles/ggml-cpu-armv9.2_2.dir/build.make:76: ggml/src/CMakeFiles/ggml-cpu-armv9.2_2.dir/ggml-cpu/ggml-cpu.c.o] Error 1
#12 198.6 gmake[2]: *** [ggml/src/CMakeFiles/ggml-cpu-armv9.2_1.dir/build.make:76: ggml/src/CMakeFiles/ggml-cpu-armv9.2_1.dir/ggml-cpu/ggml-cpu.c.o] Error 1
#12 198.6 gmake[1]: *** [CMakeFiles/Makefile2:1377: ggml/src/CMakeFiles/ggml-cpu-armv9.2_2.dir/all] Error 2
#12 198.6 gmake[1]: *** Waiting for unfinished jobs....
#12 198.6 gmake[1]: *** [CMakeFiles/Makefile2:1324: ggml/src/CMakeFiles/ggml-cpu-armv9.2_1.dir/all] Error 2

As I have discussed earlier, the solution for both CPU and CUDA is to use GCC 14 in Ubuntu 24.04:

# Add gcc-14 and g++-14
sed -i 's/apt-get install -y build-essential/apt-get install -y gcc-14 g++-14 build-essential/g' .devops/cuda-new.Dockerfile
# Add `CC=gcc-14 CXX=g++-14 CUDAHOSTCXX=g++-14`
sed -i '/apt-get install -y gcc-14 g++-14 build-essential/a ENV CC=gcc-14 CXX=g++-14 CUDAHOSTCXX=g++-14' .devops/cuda-new.Dockerfile

[Ts-sound]

I know what you mean.
But this is only a temporary solution. The issue will reoccur if new ARM64 features are added (not supported by GCC 14) .
The important thing is that we need to add the corresponding logic checks and handling in CMake.

[Ts-sound]

seems ok . @taronaeo

https://github.com/Ts-sound/llama.cpp/actions/runs/23631183339

[ehfd]

I know what you mean. But this is only a temporary solution. The issue will reoccur if new ARM64 features are added (not supported by GCC 14) . The important thing is that we need to add the corresponding logic checks and handling in CMake.

Well then, the PR that adds new ARM64 feature gates would have to add those capabilities in this repository first, wouldn't it? Unless the flags like sme are turned on by the compiler automatically somehow?

[taronaeo]

LGTM, will need 1 more approval.

[webfrogs]

LGTM