Tune gemm op bf16 #1190
Merged
Tune gemm op bf16 #1190
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* update * fix * fix2 * update * update gemm/fmoe tune * modify README.md * update gen_instances.py * fix lint error * fix typo * rm env AITER_CONFIG_** in ck gen_instances.py * update a8w8_blockscale_bpreshuffle * support to read config from multiple tuned files * fix error in batch_gemm --------- Co-authored-by: Ying.Zhou2 <Ying.Zhou2@amd.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com>
…1170) * Make bool guard return func itself * make all return func * write querysol fake
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for BF16 ASM (assembly) GEMM kernels to the tuning framework, allowing these optimized kernels to be benchmarked and included in the tuning results alongside existing hipBLAS and rocBLAS solutions.
- Integrates BF16 ASM kernels into the GEMM tuning pipeline with splitK support
- Updates data structures and CSV formats to accommodate new kernel information
- Modifies the tuned GEMM lookup system to handle ASM kernel parameters
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| gradlib/gradlib/gemm_tuner.py | Added splitK command line argument (commented out in usage) |
| gradlib/gradlib/GemmTuner.py | Core implementation adding ASM kernel tuning, updated CSV format, and enhanced result tracking |
| gradlib/README.md | Updated documentation with new CSV column descriptions |
| csrc/py_itfs_cu/asm_gemm_a16w16.cu | Modified kernel selection logic to handle optional parameters |
| aiter/tuned_gemm.py | Updated solution lookup to support ASM kernels with splitK and kernel name parameters |
| aiter/configs/untuned_gemm.csv | Added test GEMM configurations |
| aiter/configs/tuned_gemm.csv | Updated CSV format with new columns and sample tuned results |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
valarLip
reviewed
Oct 16, 2025
rm m < 256 limitation in gemm_op_a4w4.py
valarLip
approved these changes
Oct 20, 2025
eliotwang
pushed a commit
to eliotwang/aiter
that referenced
this pull request
Oct 21, 2025
* add asm bf16_gemm to gemm tuner * mdf_fileName (ROCm#1160) * Tuning Utility update (ROCm#1102) * update * fix * fix2 * update * update gemm/fmoe tune * modify README.md * update gen_instances.py * fix lint error * fix typo * rm env AITER_CONFIG_** in ck gen_instances.py * update a8w8_blockscale_bpreshuffle * support to read config from multiple tuned files * fix error in batch_gemm --------- Co-authored-by: Ying.Zhou2 <Ying.Zhou2@amd.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com> * [MI35X] Fix meta mha 950 error to make guard return bool func itself (ROCm#1170) * Make bool guard return func itself * make all return func * write querysol fake * Migrate tests to mi325-1gpu runner due to the slow network issues on TW cluster (ROCm#1166) * Temporarily using mi355 runners util issues on mi325 runners are fixed (ROCm#1176) * Update sglang_downstream.yaml to use aiter-mi325-1gpu runner (ROCm#1181) * CI: Add Aiter Release Package CI (ROCm#1171) * update gemmTuner * update tuned_gemm.py * fix lint error * update asm maxSplitK and tuned_gemm.csv * Add AITER_CONFIG_GEMM_BF16 and profile_file params to save all temp results * fix error * add padded_m logic and update gemm_common * refactor get tuned config in tuned_gemm rm m < 256 limitation in gemm_op_a4w4.py * fix lint err * wrapper the get_gemm_a16w16_config using torch_compile_guard * fix lint error * refactor tuned_gemm.py * wrapper gemm_a16w16 with torch_compile_guard * fix lint error * add gen_fake func * update * updae GemmTuner --profile_file --------- Co-authored-by: amd-ruitang3 <145657428+amd-ruitang3@users.noreply.github.com> Co-authored-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com> Co-authored-by: ZhangLirong <lirzhang@amd.com> Co-authored-by: valarLip <340077269@qq.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
add gemm bf16 asm kernel to be tuned
Technical Details
Test Plan
Test Result
Submission Checklist