Skip to content

[AMD][MI30X]Update Qwen3.5 perf#986

Merged
chunfangamd merged 6 commits intomainfrom
todd/qwen35-mi30x
Apr 15, 2026
Merged

[AMD][MI30X]Update Qwen3.5 perf#986
chunfangamd merged 6 commits intomainfrom
todd/qwen35-mi30x

Conversation

@zhentaocc
Copy link
Copy Markdown
Collaborator

@zhentaocc zhentaocc commented Apr 1, 2026 •
edited by chunfangamd
Loading

  • Added new config keys for Qwen3.5 BF16 and FP8 benchmarks on MI300X and MI325X.
  • Updated Docker image to lmsysorg/sglang:v0.5.10-rocm720-mi30x for better compatibility.
  • Enhanced benchmark scripts with additional parameters for context length and prefill tokens.
  • Adjusted memory fraction settings and added new flags for server launch to optimize performance.

e2e Test: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/24170176668

Co-Authored-by: @chunfangamd @1am9trash

@zhentaocc zhentaocc requested a review from a team April 1, 2026 03:14
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 1, 2026

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 1, 2026

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

@zhentaocc zhentaocc self-assigned this Apr 1, 2026
claude[bot]
claude Bot reviewed Apr 1, 2026
View reviewed changes
Comment thread benchmarks/single_node/qwen3.5_bf16_mi300x.sh
Comment thread perf-changelog.yaml
functionstackx
functionstackx reviewed Apr 1, 2026
View reviewed changes
Comment thread benchmarks/single_node/qwen3.5_bf16_mi325x.sh
chunfangamd
chunfangamd requested changes Apr 1, 2026
View reviewed changes
Comment thread benchmarks/single_node/qwen3.5_bf16_mi300x.sh
Comment thread benchmarks/single_node/qwen3.5_bf16_mi300x.sh
Comment thread benchmarks/single_node/qwen3.5_bf16_mi300x.sh
Comment thread benchmarks/single_node/qwen3.5_bf16_mi300x.sh
Comment thread benchmarks/single_node/qwen3.5_bf16_mi300x.sh
Comment thread benchmarks/single_node/qwen3.5_bf16_mi300x.sh
@cquil11
Copy link
Copy Markdown
Collaborator

cquil11 commented Apr 5, 2026

bump @zhentaocc

Chen, Todd added 4 commits April 8, 2026 21:38
Update Qwen3.5 benchmark configurations and scripts for improved perf…
a9e3221 
…ormance

- Added new config keys for Qwen3.5 BF16 and FP8 benchmarks on MI300X and MI325X.
- Updated Docker image to lmsysorg/sglang:v0.5.10rc0-rocm720-mi30x for better compatibility.
- Enhanced benchmark scripts with additional parameters for context length and prefill tokens.
- Adjusted memory fraction settings and added new flags for server launch to optimize performance.
Update Qwen3.5 benchmark PR link in perf-changelog.yaml
f05c916
Remove --ep-size parameter from Qwen3.5 benchmark scripts for BF16 an…
c647a86 
…d FP8 configurations on MI300X and MI325X to streamline server launch commands.
Update Qwen3.5 benchmark scripts to use 'aiter' as the attention back…
2c17191 
…end instead of 'triton' for BF16 and FP8 configurations on MI300X and MI325X.
@zhentaocc zhentaocc force-pushed the todd/qwen35-mi30x branch from e307c2d to 2c17191 Compare April 9, 2026 02:40
Update Qwen3.5 benchmark configurations to use stable Docker image ve…
970742b 
…rsion for MI300X and MI325X setups. Changed image tag from 'v0.5.10rc0-rocm720-mi30x' to 'v0.5.10-rocm720-mi30x' for consistency and reliability.
@cquil11
Copy link
Copy Markdown
Collaborator

cquil11 commented Apr 14, 2026

@chunfangamd @zhentaocc This looks good to me, what is the hold up?
CleanShot 2026-04-14 at 14 58 14

functionstackx
functionstackx requested changes Apr 15, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@functionstackx functionstackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cookbook

Comment thread benchmarks/single_node/qwen3.5_bf16_mi300x.sh
functionstackx
functionstackx approved these changes Apr 15, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@functionstackx functionstackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@zhentaocc
Copy link
Copy Markdown
Collaborator Author

zhentaocc commented Apr 15, 2026

sweep run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/24170176668

@chunfangamd chunfangamd self-requested a review April 15, 2026 06:53
chunfangamd
chunfangamd approved these changes Apr 15, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@chunfangamd chunfangamd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@chunfangamd chunfangamd merged commit 20073ba into main Apr 15, 2026
21 checks passed
@chunfangamd chunfangamd deleted the todd/qwen35-mi30x branch April 15, 2026 06:54
@claude claude Bot mentioned this pull request Apr 16, 2026
Merged
cquil11 added a commit that referenced this pull request Apr 17, 2026
@cquil11
Revert "[AMD][MI30X]Update Qwen3.5 perf (#986)" [skip-sweep]
d090c56 
This reverts commit 20073ba, except
for changes to benchmarks/single_node/qwen3.5_{bf16,fp8}_mi355x.sh,
which have been preserved to retain PR #1036's subsequent fixes.
@cquil11 cquil11 mentioned this pull request Apr 17, 2026
Merged
cquil11 added a commit that referenced this pull request Apr 17, 2026
@cquil11
Revert "[AMD][MI30X]Update Qwen3.5 perf (#986)" [skip-sweep] (#1062)
752685d 
[skip-sweep]

This reverts commit 20073ba, except for changes to benchmarks/single_node/qwen3.5_{bf16,fp8}_mi355x.sh, which have been preserved to retain PR #1036's subsequent fixes.
@cquil11 cquil11 mentioned this pull request Apr 17, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments

@functionstackx functionstackx functionstackx approved these changes

@chunfangamd chunfangamd chunfangamd approved these changes

@billishyahao billishyahao Awaiting requested review from billishyahao billishyahao is a code owner

@seungrokj seungrokj Awaiting requested review from seungrokj seungrokj is a code owner

@yctseng0211 yctseng0211 Awaiting requested review from yctseng0211 yctseng0211 is a code owner

@1am9trash 1am9trash Awaiting requested review from 1am9trash 1am9trash is a code owner

Assignees

@zhentaocc zhentaocc

Labels

AMD

Projects

InferenceMAX Board
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zhentaocc @cquil11 @functionstackx @chunfangamd