Skip to content

CUDA: fix LoRAs#3130

Merged
JohannesGaessler merged 1 commit intoggml-org:masterfrom
JohannesGaessler:cuda-fix-lora
Sep 12, 2023
Merged

CUDA: fix LoRAs#3130
JohannesGaessler merged 1 commit intoggml-org:masterfrom
JohannesGaessler:cuda-fix-lora

Conversation

@JohannesGaessler
Copy link
Copy Markdown
Contributor

@JohannesGaessler JohannesGaessler commented Sep 11, 2023

As pointed out by #3110 (comment) , the CUDA code for LoRAs was broken by #3110 . This PR fixes this.

slaren
slaren reviewed Sep 11, 2023
View reviewed changes
Comment thread ggml-cuda.cu Outdated
@JohannesGaessler
Copy link
Copy Markdown
Contributor Author

JohannesGaessler commented Sep 12, 2023

Previously ggml_cpy_tensor_2d was not called for LoRAs. As it turns out the logic for src0_on_device was incorrect so the tensor was being copied unnecessarily. I'm keeping the extended logic for ggml_cpy_tensor_2d.

@JohannesGaessler JohannesGaessler merged commit 4f7cd6b into ggml-org:master Sep 12, 2023
pkrmf pushed a commit to morlockstudios-com/llama.cpp that referenced this pull request Sep 26, 2023
@JohannesGaessler @pkrmf
CUDA: fix LoRAs (ggml-org#3130)
b958c90
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@JohannesGaessler
CUDA: fix LoRAs (ggml-org#3130)
d86076e
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
@JohannesGaessler
CUDA: fix LoRAs (ggml-org#3130)
99793f9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@slaren slaren slaren approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@JohannesGaessler @slaren