Tuning Utility update #1102

valarLip · 2025-09-27T09:33:21Z

Motivation

add env "AITER_LOG_TUNED_CONFIG" to control log for tuned shape
add env "AITER_CONFIG_GEMM_A4W4" for load specified config by User, like
- AITER_CONFIG_GEMM_A4W4=xxxx/llama_a4w4_config.csv --->for llama
- AITER_CONFIG_GEMM_A4W4=xxxx/qwen_a4w4_config.csv --->for qwen

Technical Details

Test Plan

Test Result

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Copilot

Pull Request Overview

This PR adds environment variable support for tuning utility configuration in the AITER library. It introduces two new environment variables to control logging behavior and allow users to specify custom configuration files for GEMM A4W4 operations.

Adds AITER_LOG_TUNED_CONFIG environment variable to control logging of tuned shape configurations
Adds AITER_CONFIG_GEMM_A4W4 environment variable to specify custom configuration files for GEMM A4W4 operations
Updates code organization and imports in several files to support these new features

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
csrc/ck_gemm_a4w4_blockscale/gen_instances.py	Reorganizes imports and updates default tune file path to use the new environment variable
csrc/ck_gemm_a4w4_blockscale/gemm_a4w4_blockscale_tune.py	Reorganizes imports and updates default tune file path
aiter/ops/gemm_op_a4w4.py	Adds environment variable definitions and conditional logging based on AITER_LOG_TUNED_CONFIG
aiter/jit/optCompilerConfig.json	Updates blob generation command to use the new environment variable
aiter/jit/core.py	Adds AITER_LOG_TUNED_CONFIG definition and improves type hints

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

aiter/ops/gemm_op_a4w4.py

aiter/jit/core.py

dhonnappa-amd · 2025-09-28T10:45:28Z

Jenkins CI skipped: Check lint failed. Exiting the entire job...

dhonnappa-amd · 2025-09-28T11:00:09Z

Jenkins CI skipped: Check lint failed. Exiting the entire job...

* update * fix * fix2 * update * update gemm/fmoe tune * modify README.md * update gen_instances.py * fix lint error * fix typo * rm env AITER_CONFIG_** in ck gen_instances.py * update a8w8_blockscale_bpreshuffle * support to read config from multiple tuned files * fix error in batch_gemm --------- Co-authored-by: Ying.Zhou2 <Ying.Zhou2@amd.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com>

* add asm bf16_gemm to gemm tuner * mdf_fileName (#1160) * Tuning Utility update (#1102) * update * fix * fix2 * update * update gemm/fmoe tune * modify README.md * update gen_instances.py * fix lint error * fix typo * rm env AITER_CONFIG_** in ck gen_instances.py * update a8w8_blockscale_bpreshuffle * support to read config from multiple tuned files * fix error in batch_gemm --------- Co-authored-by: Ying.Zhou2 <Ying.Zhou2@amd.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com> * [MI35X] Fix meta mha 950 error to make guard return bool func itself (#1170) * Make bool guard return func itself * make all return func * write querysol fake * Migrate tests to mi325-1gpu runner due to the slow network issues on TW cluster (#1166) * Temporarily using mi355 runners util issues on mi325 runners are fixed (#1176) * Update sglang_downstream.yaml to use aiter-mi325-1gpu runner (#1181) * CI: Add Aiter Release Package CI (#1171) * update gemmTuner * update tuned_gemm.py * fix lint error * update asm maxSplitK and tuned_gemm.csv * Add AITER_CONFIG_GEMM_BF16 and profile_file params to save all temp results * fix error * add padded_m logic and update gemm_common * refactor get tuned config in tuned_gemm rm m < 256 limitation in gemm_op_a4w4.py * fix lint err * wrapper the get_gemm_a16w16_config using torch_compile_guard * fix lint error * refactor tuned_gemm.py * wrapper gemm_a16w16 with torch_compile_guard * fix lint error * add gen_fake func * update * updae GemmTuner --profile_file --------- Co-authored-by: amd-ruitang3 <145657428+amd-ruitang3@users.noreply.github.com> Co-authored-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com> Co-authored-by: ZhangLirong <lirzhang@amd.com> Co-authored-by: valarLip <340077269@qq.com>

* update * fix * fix2 * update * update gemm/fmoe tune * modify README.md * update gen_instances.py * fix lint error * fix typo * rm env AITER_CONFIG_** in ck gen_instances.py * update a8w8_blockscale_bpreshuffle * support to read config from multiple tuned files * fix error in batch_gemm --------- Co-authored-by: Ying.Zhou2 <Ying.Zhou2@amd.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com>

* add asm bf16_gemm to gemm tuner * mdf_fileName (ROCm#1160) * Tuning Utility update (ROCm#1102) * update * fix * fix2 * update * update gemm/fmoe tune * modify README.md * update gen_instances.py * fix lint error * fix typo * rm env AITER_CONFIG_** in ck gen_instances.py * update a8w8_blockscale_bpreshuffle * support to read config from multiple tuned files * fix error in batch_gemm --------- Co-authored-by: Ying.Zhou2 <Ying.Zhou2@amd.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com> * [MI35X] Fix meta mha 950 error to make guard return bool func itself (ROCm#1170) * Make bool guard return func itself * make all return func * write querysol fake * Migrate tests to mi325-1gpu runner due to the slow network issues on TW cluster (ROCm#1166) * Temporarily using mi355 runners util issues on mi325 runners are fixed (ROCm#1176) * Update sglang_downstream.yaml to use aiter-mi325-1gpu runner (ROCm#1181) * CI: Add Aiter Release Package CI (ROCm#1171) * update gemmTuner * update tuned_gemm.py * fix lint error * update asm maxSplitK and tuned_gemm.csv * Add AITER_CONFIG_GEMM_BF16 and profile_file params to save all temp results * fix error * add padded_m logic and update gemm_common * refactor get tuned config in tuned_gemm rm m < 256 limitation in gemm_op_a4w4.py * fix lint err * wrapper the get_gemm_a16w16_config using torch_compile_guard * fix lint error * refactor tuned_gemm.py * wrapper gemm_a16w16 with torch_compile_guard * fix lint error * add gen_fake func * update * updae GemmTuner --profile_file --------- Co-authored-by: amd-ruitang3 <145657428+amd-ruitang3@users.noreply.github.com> Co-authored-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Co-authored-by: Xin Huang <Xin.Huang@amd.com> Co-authored-by: ZhangLirong <lirzhang@amd.com> Co-authored-by: valarLip <340077269@qq.com>

update

829224a

Copilot AI review requested due to automatic review settings September 27, 2025 09:33

Copilot AI reviewed Sep 27, 2025

View reviewed changes

aiter/ops/gemm_op_a4w4.py Show resolved Hide resolved

aiter/jit/core.py Show resolved Hide resolved

valarLip and others added 4 commits September 27, 2025 17:42

fix

d8ea78c

fix2

2c1161d

update

4b3f16d

update gemm/fmoe tune

9cfd7f0

Merge branch 'main' into tuning_utilily_update

6124852

yzhou103 and others added 6 commits September 28, 2025 19:30

modify README.md

039730e

update gen_instances.py

15c9f2f

fix lint error

e7992fd

Merge branch 'main' into tuning_utilily_update

2bdbe9b

fix typo

a3d6569

Merge branch 'main' into tuning_utilily_update

7a8fdc1

valarLip mentioned this pull request Sep 29, 2025

[config] tune gemm and moe for qwen3 480b ptpc model on MI308 #1084

Open

yzhou103 and others added 8 commits September 29, 2025 16:53

Merge branch 'main' into tuning_utilily_update

16680cf

rm env AITER_CONFIG_** in ck gen_instances.py

a4ecc04

update a8w8_blockscale_bpreshuffle

265152b

support to read config from multiple tuned files

e9d0d36

fix error in batch_gemm

6717880

Merge branch 'main' into tuning_utilily_update

01c1607

Merge branch 'main' into tuning_utilily_update

845282d

Merge branch 'main' into tuning_utilily_update

9e74a1f

valarLip self-assigned this Oct 10, 2025

yzhou103 and others added 2 commits October 11, 2025 10:37

Merge branch 'main' into tuning_utilily_update

70910f8

Merge branch 'main' into tuning_utilily_update

6f82850

valarLip merged commit ddcc4ea into main Oct 12, 2025
16 of 17 checks passed

valarLip deleted the tuning_utilily_update branch October 12, 2025 07:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Tuning Utility update #1102

Tuning Utility update #1102

Uh oh!

valarLip commented Sep 27, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

dhonnappa-amd commented Sep 28, 2025

Uh oh!

dhonnappa-amd commented Sep 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Tuning Utility update #1102

Tuning Utility update #1102

Uh oh!

Conversation

valarLip commented Sep 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

dhonnappa-amd commented Sep 28, 2025

Uh oh!

dhonnappa-amd commented Sep 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

valarLip commented Sep 27, 2025 •

edited

Loading