gpt-oss 120B — H100 vs MI300X
Head-to-head AI inference benchmark comparison of H100 (NVIDIA Hopper) and MI300X (AMD CDNA 3) on gpt-oss 120B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Setting 112 tok/s/user as the target on gpt-oss 120B, H100 produces 2796 tok/s/chip ($0.12 per million tokens) and MI300X produces 1599 ($0.17). H100 is 41% cheaper per token; H100 delivers 75% more tok/s/chip.
At 158 tok/s/user interactivity on gpt-oss 120B, H100 delivers 1521 tok/s/chip at $0.21 per million tokens; MI300X delivers 803 tok/s/chip at $0.32. H100 is 48% cheaper per token; H100 delivers 90% more tok/s/chip at this point.
H100 posts 878 tok/s/chip for $0.37 per million tokens at 203 tok/s/user on gpt-oss 120B; MI300X posts 309 tok/s/chip for $0.83. H100 is 125% cheaper per token; H100 delivers 184% more tok/s/chip. (Numbers reflect the default 1k/1k · fp4 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/chip) | H100:2795.7MI300X:1598.8 | H100:1521.3MI300X:802.8 | H100:878.3MI300X:309.1 |
| Cost ($/M tok) | H100:$0.117MI300X:$0.166 | H100:$0.214MI300X:$0.316 | H100:$0.368MI300X:$0.827 |
| tok/s/MW | H100:2040676MI300X:1150184 | H100:1110436MI300X:577534 | H100:641118MI300X:222381 |
| Concurrency | H100:~64MI300X:~15 | H100:~28MI300X:~5 | H100:~10MI300X:~7 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.