Skip to content

[kernel] add fused_qk_rmsnorm_per_token_quant kernel#2958

Open
gbyu-amd wants to merge 5 commits intomainfrom
guanbao/fuse_qknorm_per_token_quant
Open

[kernel] add fused_qk_rmsnorm_per_token_quant kernel#2958
gbyu-amd wants to merge 5 commits intomainfrom
guanbao/fuse_qknorm_per_token_quant

Conversation

@gbyu-amd
Copy link
Copy Markdown
Contributor

Motivation

Some quark models, e.g., amd/DeepSeek-R1-0528-MXFP4-MTP-MoEFP4 and amd/Kimi-K2-Thinking-MXFP4-AttnFP8 have fp8 weight linear layers in attn and adopt ptpc quant recipe, thus add fused_qk_rmsnorm_per_token_quant kernel in this pr which will be used in ATOM/vLLM-ATOM.

Technical Details

Test Plan

Test Result

Submission Checklist

@gbyu-amd gbyu-amd requested a review from a team April 29, 2026 09:39
@github-actions
Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests
ci:atom ATOM benchmark (DeepSeek-R1 + GPT-OSS)
ci:vllm vLLM benchmark
ci:all All of the above

Add labels via the sidebar or gh pr edit 2958 --add-label <label>

@gbyu-amd gbyu-amd marked this pull request as draft April 29, 2026 11:46
@gbyu-amd gbyu-amd marked this pull request as ready for review April 29, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant