Add support for TRUNC unary operator (CPU +SYCL)#16032
Add support for TRUNC unary operator (CPU +SYCL)#16032safranowith wants to merge 6 commits intoggml-org:masterfrom
Conversation
github-actions
Bot
added
documentation
ggerganov
left a comment
ggerganov
left a comment
There was a problem hiding this comment.
It might be better to consolidate all new unary ops PRs that are submitted recently into 2 PRs:
- One with CPU-only implementation
- One with SYCL implementation
The SYCL should be reviewed by @NeoZhangJianyu or someone else on the Intel team (not sure who is active these days).
|
|
||
| for (const std::string& btype : {"f16", "f32", "q8_1"}) { | ||
| std::vector<std::string> btypes = {"f16", "f32"}; | ||
|
|
||
| #if defined(GGML_VULKAN_INTEGER_DOT_GLSLC_SUPPORT) | ||
| btypes.push_back("q8_1"); | ||
| #endif | ||
|
|
||
| for (const std::string& btype : btypes) { | ||
| for (const auto& tname : type_names) { |
There was a problem hiding this comment.
This change should be in a separate PR
There was a problem hiding this comment.
This is my code from #15740, no idea what it is doing here.
There was a problem hiding this comment.
I don't understand what needs fixing
This is not a change I made
I would appreciate a clearer explanation
thanks
There was a problem hiding this comment.
The change is in 75faa5a for whatever reason, it should not be a part of that commit.
There was a problem hiding this comment.
Not fully, no. You can look this up yourself.
There was a problem hiding this comment.
Be careful not to add extra files that are not used in the project.
There was a problem hiding this comment.
Thanks for the review!
This has been fixed.
|
|
||
| static const char * GGML_UNARY_OP_NAME[GGML_UNARY_OP_COUNT] = { | ||
| "ABS", | ||
| "ROUND", |
There was a problem hiding this comment.
| "ROUND", | |
| "TRUNC", |
There was a problem hiding this comment.
The error has been corrected.
|
|
||
| enum ggml_unary_op { | ||
| GGML_UNARY_OP_ABS, | ||
| GGML_UNARY_OP_TRUNC, |
There was a problem hiding this comment.
Add new unary ops at the end of the enum.
There was a problem hiding this comment.
Thanks for the review
This will be fixed soon
There was a problem hiding this comment.
This has been fixed.
5 participants
This PR introduces the TRUNC unary operator, applying element-wise truncation (truncf / sycl::trunc) on tensors.
Changes:
CPU (ggml-cpu.c)
Added forward implementation: ggml_compute_forward_trunc
Registered op in scheduler: ggml_get_n_tasks
SYCL (ggml-sycl)
Added device function: op_trunc
Added kernel: unary_op_trunc_kernel
Added dispatch wrapper: ggml_sycl_op_trunc
Exposed API: ggml_sycl_trunc
Tests
Included in backend coverage tests
Support matrix can be regenerated:
./build/bin/test-backend-ops support -o TRUNC --output csv > trunc_support.csv
./scripts/create_ops_docs.py
Produces trunc_support.csv verifying backend availability.
Notes
Implemented for F32 (and F16 where SYCL supports it)
Brings consistency with existing unary ops