Skip to content

chore(llama.cpp): bump, fix sycl linking issues #2997

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mudler wants to merge 3 commits into from
Closed

chore(llama.cpp): bump, fix sycl linking issues #2997

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4c16957
deps(llama.cpp): bump, try to fix sycl linking issues
mudler Jul 24, 2024
f80b6df
test
mudler Jul 24, 2024
59bfc67
workaround upstream issue
mudler Jul 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ DETECT_LIBS?=true
# llama.cpp versions
GOLLAMA_REPO?=https://github.com/go-skynet/go-llama.cpp
GOLLAMA_VERSION?=2b57a8ae43e4699d3dc5d1496a1ccd42922993be
CPPLLAMA_VERSION?=081fe431aa8fb6307145c4feb3eed4f48cab19f8
CPPLLAMA_VERSION?=b841d0740855c5af1344a81f261139a45a2b39ee

# gpt4all version
GPT4ALL_REPO?=https://github.com/nomic-ai/gpt4all
Expand Down
23 changes: 18 additions & 5 deletions backend/cpp/llama/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,24 @@ add_library(hw_grpc_proto
${hw_proto_hdrs} )

add_executable(${TARGET} grpc-server.cpp utils.hpp json.hpp)
target_link_libraries(${TARGET} PRIVATE common llama myclip ${CMAKE_THREAD_LIBS_INIT} absl::flags hw_grpc_proto
absl::flags_parse
gRPC::${_REFLECTION}
gRPC::${_GRPC_GRPCPP}
protobuf::${_PROTOBUF_LIBPROTOBUF})

# Conditionally link SYCL to grpc-server
# https://github.com/ggerganov/llama.cpp/issues/8665
if ( DEFINED ENV{ONEAPI_ROOT})
target_link_libraries(${TARGET} PRIVATE common llama myclip ${CMAKE_THREAD_LIBS_INIT} absl::flags hw_grpc_proto
absl::flags_parse
gRPC::${_REFLECTION}
gRPC::${_GRPC_GRPCPP}
protobuf::${_PROTOBUF_LIBPROTOBUF}
sycl)
else()
target_link_libraries(${TARGET} PRIVATE common llama myclip ${CMAKE_THREAD_LIBS_INIT} absl::flags hw_grpc_proto
absl::flags_parse
gRPC::${_REFLECTION}
gRPC::${_GRPC_GRPCPP}
protobuf::${_PROTOBUF_LIBPROTOBUF})
endif()

target_compile_features(${TARGET} PRIVATE cxx_std_11)
if(TARGET BUILD_INFO)
add_dependencies(${TARGET} BUILD_INFO)
Expand Down
8 changes: 8 additions & 0 deletions backend/cpp/llama/CMakeLists.txt.rpc-8662
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# https://github.com/ggerganov/llama.cpp/issues/8665

add_executable(rpc-server rpc-server.cpp)
if ( DEFINED ENV{ONEAPI_ROOT})
target_link_libraries(rpc-server PRIVATE ggml llama sycl)
else()
target_link_libraries(rpc-server PRIVATE ggml llama)
endif()
5 changes: 4 additions & 1 deletion backend/cpp/llama/prepare.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,7 @@ cp -rfv llama.cpp/examples/llava/clip.h llama.cpp/examples/grpc-server/clip.h
cp -rfv llama.cpp/examples/llava/llava.cpp llama.cpp/examples/grpc-server/llava.cpp
echo '#include "llama.h"' > llama.cpp/examples/grpc-server/llava.h
cat llama.cpp/examples/llava/llava.h >> llama.cpp/examples/grpc-server/llava.h
cp -rfv llama.cpp/examples/llava/clip.cpp llama.cpp/examples/grpc-server/clip.cpp
cp -rfv llama.cpp/examples/llava/clip.cpp llama.cpp/examples/grpc-server/clip.cpp

# https://github.com/ggerganov/llama.cpp/issues/8665
cp -rfv CMakeLists.txt.rpc-8662 llama.cpp/examples/rpc/CMakeLists.txt