Qwen 3.5 397B-A17B — GB200 NVL72 vs MI300X
Head-to-head AI inference benchmark comparison of GB200 NVL72 (NVIDIA Blackwell) and MI300X (AMD CDNA 3) on Qwen 3.5 397B-A17B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Near the low end of the 46–66 tok/s/user interactivity band, at 51 tok/s/user on Qwen 3.5 397B-A17B: GB200 NVL72 runs 12378 tok/s/chip at $0.04/M tokens, MI300X runs 560 at $0.47/M. GB200 NVL72 is 1023% cheaper per token; GB200 NVL72 delivers 2112% more tok/s/chip.
Setting 56 tok/s/user as the target on Qwen 3.5 397B-A17B, GB200 NVL72 produces 12267 tok/s/chip ($0.04 per million tokens) and MI300X produces 451 ($0.59). GB200 NVL72 is 1291% cheaper per token; GB200 NVL72 delivers 2621% more tok/s/chip.
At 61 tok/s/user interactivity on Qwen 3.5 397B-A17B, GB200 NVL72 delivers 12105 tok/s/chip at $0.04 per million tokens; MI300X delivers 359 tok/s/chip at $0.75. GB200 NVL72 is 1668% cheaper per token; GB200 NVL72 delivers 3272% more tok/s/chip at this point. (Numbers reflect the default 8k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/chip) | GB200 NVL72:12378.1MI300X:559.5 | GB200 NVL72:12267.1MI300X:450.9 | GB200 NVL72:12104.7MI300X:359.0 |
| Cost ($/M tok) | GB200 NVL72:$0.042MI300X:$0.469 | GB200 NVL72:$0.042MI300X:$0.586 | GB200 NVL72:$0.043MI300X:$0.755 |
| tok/s/MW | GB200 NVL72:6619279MI300X:402536 | GB200 NVL72:6559922MI300X:324354 | GB200 NVL72:6473107MI300X:258243 |
| Concurrency | GB200 NVL72:~1899MI300X:~11 | GB200 NVL72:~1609MI300X:~8 | GB200 NVL72:~1443MI300X:~6 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.