Skip to content

[AMD/ROCM] GLM5/5.1 FP8 MTP Support on MI355X#1122

Draft
ajith-sirra-amd wants to merge 16 commits intoSemiAnalysisAI:mainfrom
ajith-sirra-amd:glm5_fp8_mtp_mi355x_sglang
Draft

[AMD/ROCM] GLM5/5.1 FP8 MTP Support on MI355X#1122
ajith-sirra-amd wants to merge 16 commits intoSemiAnalysisAI:mainfrom
ajith-sirra-amd:glm5_fp8_mtp_mi355x_sglang

Conversation

@ajith-sirra-amd
Copy link
Copy Markdown
Collaborator

@ajith-sirra-amd ajith-sirra-amd commented Apr 23, 2026

Overview

Add GLM-5 (GLM5.1 architecture) FP8 MTP benchmark configuration and testing support for AMD MI355X hardware.

Changes

  • Added benchmark script for GLM-5.1 FP8 model on MI355X with MTP to run with Updated SGLang Image.
  • Updated GitHub Actions configuration for AMD Master Yaml File.

Testing

  • Verify benchmark execution on MI355X hardware
  • Validate configuration settings

Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
@seungrokj
Copy link
Copy Markdown
Collaborator

/sweep test-config --config-files .github/configs/amd-master.yaml --config-keys glm5.1-fp8-mi355x-sglang-mtp

@github-actions
Copy link
Copy Markdown
Contributor

@seungrokj Kicking off a sweep.

Run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/24835029786
Command: test-config --config-files .github/configs/amd-master.yaml --config-keys glm5.1-fp8-mi355x-sglang-mtp
Pinned ref: 5a9c062
Approval: not required (trusted collaborator).

Copy link
Copy Markdown
Contributor

@functionstackx functionstackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can u remove glm5.1 then? glm5.1 & glm5 is in the same class of architecture

#1086

@seungrokj
Copy link
Copy Markdown
Collaborator

@ajith-sirra-amd can you plz update glm5.1 to glm5 (so that this PR is an TP4 search space extension of existing PR #1086) ?

@seungrokj seungrokj added the AMD label Apr 24, 2026
Copy link
Copy Markdown
Collaborator

@chunfangamd chunfangamd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Comment thread perf-changelog.yaml Outdated
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
@chunfangamd chunfangamd self-requested a review April 30, 2026 07:27
Copy link
Copy Markdown
Collaborator

@chunfangamd chunfangamd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ajith-sirra-amd ajith-sirra-amd changed the title [AMD/ROCM] GLM5.1 FP8 MTP Support on MI355X [AMD/ROCM] GLM5/5.1 FP8 MTP Support on MI355X Apr 30, 2026
Copy link
Copy Markdown
Contributor

@functionstackx functionstackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Can u add back chat templates as that more closely aligns the AR distribution with real world

Comment thread benchmarks/single_node/glm5_fp8_mi355x_mtp.sh
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Copy link
Copy Markdown
Contributor

@functionstackx functionstackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, assuming there is an validation run

@SemiAnalysisAI SemiAnalysisAI deleted a comment from github-actions Bot May 1, 2026
@SemiAnalysisAI SemiAnalysisAI deleted a comment from github-actions Bot May 1, 2026
@seungrokj
Copy link
Copy Markdown
Collaborator

/sweep test-config --config-files .github/configs/amd-master.yaml --config-keys glm5-fp8-mi355x-sglang-mtp --evals-only

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

@seungrokj Kicking off a sweep.

Run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/25197000070
Command: test-config --config-files .github/configs/amd-master.yaml --config-keys glm5-fp8-mi355x-sglang-mtp --evals-only
Pinned ref: 8504624
Approval: not required (trusted collaborator).

@chunfangamd chunfangamd marked this pull request as draft May 1, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

4 participants