Qwen 3.5 397B-A17B — GB300 NVL72 vs H200
Head-to-head AI inference benchmark comparison of GB300 NVL72 (NVIDIA Blackwell) and H200 (NVIDIA Hopper) on Qwen 3.5 397B-A17B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Near the low end of the 48–187 tok/s/user interactivity band, at 83 tok/s/user on Qwen 3.5 397B-A17B: GB300 NVL72 runs 11181 tok/s/chip at $0.06/M tokens, H200 runs 1451 at $0.23/M. GB300 NVL72 is 307% cheaper per token; GB300 NVL72 delivers 670% more tok/s/chip.
Setting 118 tok/s/user as the target on Qwen 3.5 397B-A17B, GB300 NVL72 produces 6855 tok/s/chip ($0.10 per million tokens) and H200 produces 1143 ($0.30). GB300 NVL72 is 211% cheaper per token; GB300 NVL72 delivers 500% more tok/s/chip.
At 153 tok/s/user interactivity on Qwen 3.5 397B-A17B, GB300 NVL72 delivers 3752 tok/s/chip at $0.17 per million tokens; H200 delivers 925 tok/s/chip at $0.36. GB300 NVL72 is 117% cheaper per token; GB300 NVL72 delivers 306% more tok/s/chip at this point. (Numbers reflect the default 8k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/chip) | GB300 NVL72:11181.2H200:1451.2 | GB300 NVL72:6854.8H200:1142.8 | GB300 NVL72:3752.4H200:925.0 |
| Cost ($/M tok) | GB300 NVL72:$0.057H200:$0.234 | GB300 NVL72:$0.096H200:$0.298 | GB300 NVL72:$0.168H200:$0.365 |
| tok/s/MW | GB300 NVL72:5274168H200:1059296 | GB300 NVL72:3233394H200:834174 | GB300 NVL72:1769990H200:675190 |
| Concurrency | GB300 NVL72:~1209H200:~16 | GB300 NVL72:~290H200:~9 | GB300 NVL72:~59H200:~6 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.