Update ggml_sycl_op_mul_mat_vec_q#5502
Conversation
|
@NeoZhangJianyu, @abhilash1910, @Alcpz, feedback would be appreciated |
Alcpz
left a comment
There was a problem hiding this comment.
Minor comment on the refactor. Looks great.
|
Thanks @AidanBeltonS , could you please rebase to latest master for CI build? |
b7cc0f2 to
4098314
Compare
|
@ggerganov @0cc4m I think the vulkan build CI is exiting abruptly - maybe issue is common for other requests. Could you help take a look ? Thanks |
|
It's because we enabled the We can easily disable the |
|
@AidanBeltonS could you please rebase to latest master - should solve some build issues with vulkan ci. |
|
Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
b1edf62 to
5d7d353
Compare
|
* Update ggml_sycl_op_mul_mat_vec_q * Apply suggestions from code review Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com> * revert suggestion on macro * fix bug * Add quant type GGML_TYPE_IQ1_S to unsupported * fix format --------- Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
* Update ggml_sycl_op_mul_mat_vec_q * Apply suggestions from code review Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com> * revert suggestion on macro * fix bug * Add quant type GGML_TYPE_IQ1_S to unsupported * fix format --------- Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
* Update ggml_sycl_op_mul_mat_vec_q * Apply suggestions from code review Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com> * revert suggestion on macro * fix bug * Add quant type GGML_TYPE_IQ1_S to unsupported * fix format --------- Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
* Update ggml_sycl_op_mul_mat_vec_q * Apply suggestions from code review Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com> * revert suggestion on macro * fix bug * Add quant type GGML_TYPE_IQ1_S to unsupported * fix format --------- Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
This PR updates the unsupported quantized data types and refactors the code for
ggml_sycl_op_mul_mat_vec_q.SYCL does not currently have the intrinsics to support some quantized data types, this adds one missing quantized data type to the unsupported check, so tests won't be run.
This also refactors the code so there is a single templated
mul_mat_vec_q_sycl_submitterrather than multiple duplicate functions which submit a different instantiated kernel. This makes the code less verbose and much smaller.