Add DeepSeek-V4-Pro FP8 MI355X SGLang benchmark#1134

Merged

Oseltamivir merged 27 commits intomainfrom

dsv4-fp8-mi355x-sglang

Apr 24, 2026

Commits on Apr 24, 2026

Add DeepSeek-V4-Flash FP8 MI355X SGLang benchmark

Oseltamivir
and
claude
committed
Update perf-changelog pr-link to #1134

Oseltamivir
and
claude
committed
Install transformers with deepseek_v4 support in dsv4 benchmark

Oseltamivir
and
claude
committed
Also upgrade huggingface_hub to fix rope_theta type validation

Oseltamivir
and
claude
committed
Patch rope_theta int→float in downloaded config.json

Oseltamivir
and
claude
committed
Fix huggingface_hub typer clash and config.json corruption

Oseltamivir
and
claude
committed
Use huggingface_hub scan_cache_dir to find config.json for patching

Oseltamivir
and
claude
committed
Patch DeepseekV4Config source to accept int for rope_theta

Oseltamivir
and
claude
committed
Use --no-deps for transformers install; patch rope_theta before download

Oseltamivir
and
claude
committed
import error
Oseltamivir
committed
Fix Union import in rope_theta patch and allow full transformers deps

Oseltamivir
and
claude
committed
Remove corrupted cached config.json before model download

Oseltamivir
and
claude
committed
Patch all float fields in DeepseekV4Config, not just rope_theta

Oseltamivir
and
claude
committed
Use the correct SGLang image for DSv4; drop transformers patching

Oseltamivir
and
claude
committed
Patch config.json model_type so AutoConfig can load DSv4

Oseltamivir
and
claude
committed
Bump watchdog timeout to 1800s for DSv4 MoE JIT compile

Oseltamivir
and
claude
committed
Use newest SGLang DSv4 image from AMD team

Oseltamivir
and
claude
committed
Switch DSv4 benchmark from Flash to Pro

Oseltamivir
and
claude
committed
Switch dsv4pro to TP=8
Oseltamivir
committed
Revert accidental TP changes outside dsv4pro
Oseltamivir
committed
Pin max-total-tokens and raise mem-fraction-static
Oseltamivir
committed
Merge branch 'main' into dsv4-fp8-mi355x-sglang
Oseltamivir
authored
Size max-total-tokens for concurrent requests
Oseltamivir
committed
Rename dsv4pro to dsv4; bump mem-fraction-static to 0.95
Oseltamivir
committed
oom
Oseltamivir
committed
Merge branch 'main' into dsv4-fp8-mi355x-sglang
Oseltamivir
authored
Update benchmark description for DeepSeek-V4-Pro
Oseltamivir
authored

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DeepSeek-V4-Pro FP8 MI355X SGLang benchmark#1134

Add DeepSeek-V4-Pro FP8 MI355X SGLang benchmark#1134
Oseltamivir merged 27 commits intomainfrom
dsv4-fp8-mi355x-sglang

Commits on Apr 24, 2026

Add DeepSeek-V4-Flash FP8 MI355X SGLang benchmark

Update perf-changelog pr-link to #1134

Install transformers with deepseek_v4 support in dsv4 benchmark

Also upgrade huggingface_hub to fix rope_theta type validation

Patch rope_theta int→float in downloaded config.json

Fix huggingface_hub typer clash and config.json corruption

Use huggingface_hub scan_cache_dir to find config.json for patching

Patch DeepseekV4Config source to accept int for rope_theta

Use --no-deps for transformers install; patch rope_theta before download

import error

Fix Union import in rope_theta patch and allow full transformers deps

Remove corrupted cached config.json before model download

Patch all float fields in DeepseekV4Config, not just rope_theta

Use the correct SGLang image for DSv4; drop transformers patching

Patch config.json model_type so AutoConfig can load DSv4

Bump watchdog timeout to 1800s for DSv4 MoE JIT compile

Use newest SGLang DSv4 image from AMD team

Switch DSv4 benchmark from Flash to Pro

Switch dsv4pro to TP=8

Revert accidental TP changes outside dsv4pro

Pin max-total-tokens and raise mem-fraction-static

Merge branch 'main' into dsv4-fp8-mi355x-sglang

Size max-total-tokens for concurrent requests

Rename dsv4pro to dsv4; bump mem-fraction-static to 0.95

oom

Merge branch 'main' into dsv4-fp8-mi355x-sglang

Update benchmark description for DeepSeek-V4-Pro