gpt-oss 120B — H100 vs H200
Head-to-head AI inference benchmark comparison of H100 (NVIDIA Hopper) and H200 (NVIDIA Hopper) on gpt-oss 120B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Throughput at 117 tok/s/user on gpt-oss 120B: H100 hits 2622 tok/s/chip, H200 hits 2827. Per-million costs land at $0.13 and $0.12 respectively. H200 is 4% cheaper per token; H200 delivers 8% more tok/s/chip.
H100 / H200 on gpt-oss 120B at 166 tok/s/user: 1379 / 1567 tok/s/chip, $0.24 / $0.22 per million tokens. H200 is 9% cheaper per token; H200 delivers 14% more tok/s/chip.
Toward the upper edge of the 67–266 tok/s/user interactivity band, at 216 tok/s/user on gpt-oss 120B: H100 runs 740 tok/s/chip at $0.44/M tokens, H200 runs 810 at $0.42/M. H200 is 4% cheaper per token; H200 delivers 9% more tok/s/chip. (Numbers reflect the default 1k/1k · fp4 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/chip) | H100:2621.5H200:2827.3 | H100:1379.3H200:1567.1 | H100:739.7H200:809.8 |
| Cost ($/M tok) | H100:$0.125H200:$0.120 | H100:$0.236H200:$0.216 | H100:$0.438H200:$0.419 |
| tok/s/MW | H100:1913517H200:2063728 | H100:1006814H200:1143893 | H100:539935H200:591121 |
| Concurrency | H100:~64H200:~64 | H100:~17H200:~43 | H100:~8H200:~8 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.