Skip to content

[SYCL] Remove condition in mmvq#6532

Merged
NeoZhangJianyu merged 1 commit intoggml-org:masterfrom
abhilash1910:mmvq_cond
Apr 8, 2024
Merged

[SYCL] Remove condition in mmvq#6532
NeoZhangJianyu merged 1 commit intoggml-org:masterfrom
abhilash1910:mmvq_cond

Conversation

@abhilash1910
Copy link
Copy Markdown
Contributor

Remove row==1 condition in mmvq
@NeoZhangJianyu @airMeng

@airMeng
Copy link
Copy Markdown
Contributor

airMeng commented Apr 8, 2024

https://github.com/ggerganov/llama.cpp/blob/855f54402e866ed19d8d675b56a81c844c64b325/ggml-cuda.cu#L1866-L1943

can you align the whole dispatch with cuda implementation in this PR?

@abhilash1910
Copy link
Copy Markdown
Contributor Author

https://github.com/ggerganov/llama.cpp/blob/855f54402e866ed19d8d675b56a81c844c64b325/ggml-cuda.cu#L1866-L1943

can you align the whole dispatch with cuda implementation in this PR?

Yes I had tried that in a previous PR, but it seems to affect performance. Hence to keep changes minimal , I think we can keep this logic. Had discussed this before with @NeoZhangJianyu .

@NeoZhangJianyu NeoZhangJianyu merged commit 87fb5b4 into ggml-org:master Apr 8, 2024
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants