[AMD/ROCM] GLM5/5.1 FP8 MTP Support on MI355X#1122
[AMD/ROCM] GLM5/5.1 FP8 MTP Support on MI355X#1122ajith-sirra-amd wants to merge 16 commits intoSemiAnalysisAI:mainfrom
Conversation
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
|
/sweep test-config --config-files .github/configs/amd-master.yaml --config-keys glm5.1-fp8-mi355x-sglang-mtp |
|
@seungrokj Kicking off a sweep. Run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/24835029786 |
functionstackx
left a comment
There was a problem hiding this comment.
can u remove glm5.1 then? glm5.1 & glm5 is in the same class of architecture
|
@ajith-sirra-amd can you plz update glm5.1 to glm5 (so that this PR is an TP4 search space extension of existing PR #1086) ? |
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
…-sirra-amd/InferenceX into glm5_fp8_mtp_mi355x_sglang
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
functionstackx
left a comment
There was a problem hiding this comment.
Thanks for the contribution! Can u add back chat templates as that more closely aligns the AR distribution with real world
Signed-off-by: ajith-sirra-amd <ajith.sirra@amd.com>
functionstackx
left a comment
There was a problem hiding this comment.
lgtm, assuming there is an validation run
|
/sweep test-config --config-files .github/configs/amd-master.yaml --config-keys glm5-fp8-mi355x-sglang-mtp --evals-only |
|
@seungrokj Kicking off a sweep. Run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/25197000070 |
Overview
Add GLM-5 (GLM5.1 architecture) FP8 MTP benchmark configuration and testing support for AMD MI355X hardware.
Changes
Testing