add B200 Qwen SGLang BF16

[functionstackx]

@claude look at benchmarks/dsr1_fp8_b200.sh and nvidia-master.yaml for deepseek sglang fp8 b200 and then create an new benchmarks/.sh for qwen and then update nvidia-master.yaml

example command

PYTHONNOUSERSITE=1 python -m sglang.launch_server \
--model-path=$MODEL --host=0.0.0.0 --port=$PORT \
  --model Qwen/Qwen3.5-397B-A17B \
  --tp 8 \
  --mem-fraction-static 0.8

here is the recipe documentation link https://cookbook.sglang.io/autoregressive/Qwen/Qwen3.5#52-speed-benchmark. it says that the container should be lmsysorg/sglang:nightly-dev-20260216-d3bae71e

dont do stuff like this

Default: recv every ~10 requests; if CONC ≥ 16, relax to ~30 requests between scheduler recv polls.

if [[ $TP -eq 8 ]]; then
if [[ $CONC -ge 16 ]]; then
SCHEDULER_RECV_INTERVAL=30
else
SCHEDULER_RECV_INTERVAL=10
fi

Setting these values (passed in to --cuda-graph-max-bs and --max-running-requests) as the maximum concurrency

this will help us save memory from being unnecessary used.

MAX_RUNNING_REQUESTS=128
CUDA_GRAPH_MAX_BATCH_SIZE=128

MEM_FRAC_STATIC=0.82
CHUNKED_PREFILL_SIZE=32768
MAX_PREFILL_TOKENS=32768
elif [[ $TP -eq 4 ]]; then
if [[ $ISL -ne 8192 ]] || [[ $OSL -ne 1024 ]]; then
echo "TP=4 not yet supported for ISL=$ISL OSL=$OSL!"
exit 1
fi

Setting these values (passed in to --cuda-graph-max-bs and --max-running-requests) as the maximum concurrency

this will help us save memory from being unnecessary used.

MAX_RUNNING_REQUESTS=32
CUDA_GRAPH_MAX_BATCH_SIZE=32

MEM_FRAC_STATIC=0.95
CHUNKED_PREFILL_SIZE=8192
MAX_PREFILL_TOKENS=8192

SCHEDULER_RECV_INTERVAL=10
else
echo "Unrecognized TP size $TP!"
exit 1
fi
echo "SCHEDULER_RECV_INTERVAL: $SCHEDULER_RECV_INTERVAL, CONC: $CONC, ISL: $ISL, OSL: $OSL"

[Klaud-Cold]

Claude finished @functionstackx's task in 4m 24s —— View job • claude/issue-702-20260216-1843 • Create PR ➔

---

Adding Qwen 3.5 SGLang FP8 B200 Benchmark

• Read existing benchmarks/dsr1_fp8_b200.sh and nvidia-master.yaml
• Fetch SGLang Qwen recipe documentation from cookbook.sglang.io
• Analyze benchmark script naming convention and patterns
• Create benchmarks/qwen3.5_fp8_b200.sh
• Update nvidia-master.yaml with qwen3.5-fp8-b200-sglang config
• Update perf-changelog.yaml
• Commit and push changes

Summary

Created the following files/changes:

benchmarks/qwen3.5_fp8_b200.sh - Simple SGLang benchmark script (no complex TP branching as requested):

• SGLANG_USE_CUDA_IPC_TRANSPORT=1 environment variable
• --mem-fraction-static 0.8
• --reasoning-parser qwen3 for Qwen3-specific reasoning parsing
• NEXTN speculative decoding: --speculative-algo NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4
• Flags sourced from the SGLang Qwen3.5 recipe

nvidia-master.yaml - Added qwen3.5-fp8-b200-sglang config:

• Image: lmsysorg/sglang:nightly-dev-20260216-d3bae71e
• Model: Qwen/Qwen3.5-397B-A17B
• TP=8, concurrency 4-64 for 1k1k, 1k8k, and 8k1k

perf-changelog.yaml - Added changelog entry for the new config.

|

[functionstackx]

@claude dont use MTP for now, we can add MTP later. please update.

i.e. no need for SGLANG_USE_CUDA_IPC_TRANSPORT=1 or --reasoning-parser qwen3 or -speculative-algo NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4

[Klaud-Cold]

Claude finished @functionstackx's task in 1m 52s —— View job • claude/issue-702-20260216-1859

---

Removing MTP/Speculative Decoding Flags from Qwen3.5 Benchmark

• Read current benchmarks/qwen3.5_fp8_b200.sh
• Remove SGLANG_USE_CUDA_IPC_TRANSPORT=1
• Remove --reasoning-parser qwen3
• Remove speculative decoding flags (--speculative-algo NEXTN, etc.)
• Update perf-changelog.yaml
• Commit and push changes

Summary

Updated benchmarks/qwen3.5_fp8_b200.sh to remove all MTP-related flags as requested:

• Removed export SGLANG_USE_CUDA_IPC_TRANSPORT=1
• Removed --reasoning-parser qwen3
• Removed --speculative-algo NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4

The script now launches SGLang with just the basic flags: --model-path, --host, --port, --tensor-parallel-size, and --mem-fraction-static 0.8.

The nvidia-master.yaml config (qwen3.5-fp8-b200-sglang) remains unchanged — no MTP flags were there.

Create PR

View job • claude/issue-702-20260216-1859