cmake: add explicit metal version options

[mattjcly]

Currently, if you want to build llama.cpp binaries with metal enabled, you can only compile the metal libraries to be compatible with the minimum mac os version and metal shading language standard associated with your current machines SDK. This is the default behavior of of the xcrun metal command.

This presents a problem if you ever want to build llama.cpp from a newer machine (say a Mac running macOS 14.x with default metal shading language 3.1), and have it be able to run on an older machine (say a Mac running macOS 13.x with default metal shading language 3.0).

With the current state of llama.cpp, trying to perform the exact build above from macOS 14.x and running on macOS 13.x results in the following error during model load:

ggml_metal_init: error: load pipeline error: Error Domain=AGXMetal13_3 Code=3 "Function kernel_add is using language version 3.1 which is incompatible with this OS." UserInfo={NSLocalizedDescription=Function kernel_add is using language version 3.1 which is incompatible with this OS.}

By providing the new CMake options LLAMA_METAL_MACOSX_VERSION_MIN and LLAMA_METAL_STD, you can now explicitly set the macOS min version and metal shading library standard that your build will support:

cmake -DLLAMA_METAL_MACOSX_VERSION_MIN="13.6" -DLLAMA_METAL_STD="metal3.0" ..

and you will be notified in the output of that cmake command:

~/Workspace/forks/llama.cpp/build git:[mattjcly/explicit-metal-version-options]
cmake -DLLAMA_METAL_MACOSX_VERSION_MIN="13.6" -DLLAMA_METAL_STD="metal3.0" ..
-- Accelerate framework found
-- Metal framework found
-- Adding -mmacosx-version-min=13.6 flag to metal compilation
-- Adding -std=metal3.0 flag to metal compilation
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with LLAMA_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: arm64
-- ARM detected
-- Configuring done (0.1s)
-- Generating done (0.2s)
-- Build files have been written to: /Users/matt/Workspace/forks/llama.cpp/build

Doing this allows for both macOS 13.x and macOS 14.x to run llama.cpp binaries built from macOS 14.x without metal incompatibility errors.

The default behavior of builds without those options set will remain the same (since default value of these variables is "", and therefore will be ignored in the logic to add the flags to XC_FLAGS).

[phymbert]

@ggerganov Hi Georgi, I dont understand why the Benchmark job is failing since yesterday. It looks it cannot access PR comments. Any idea ?

[ggerganov]

Hm, not sure. I haven't made any changes to the runner. Does it matter if the branch is in this repo or in an external repo? I guess it shouldn't, but could be related?

[ggerganov]

It might be because I closed the terminal session to the node and some of the processes died?

I just tried to start the script with nohup:

nohup ./start-github-runner-manager.sh ggerganov/llama.cpp  ... &

But this does not seem to work. The error in the logs is:

$ cat download_model.ggml-model-q4_0.gguf.log 
the input device is not a TTY

Any suggestions? I've restarted it now, but I can't keep the session open all the time

[phymbert]

It might be because I closed the terminal session to the node and some of the processes died?
Any suggestions? I've restarted it now, but I can't keep the session open all the time

No I dont think it comes from the VM but github actions. I will investigate tom