H200 141GB

The NVIDIA H200 141GB is an upgraded variant of the H100 that replaces the 80GB HBM3 memory with 141GB of HBM3e, increasing both capacity and bandwidth. It uses the same GH100 die and Hopper architecture as the H100, delivering identical compute throughput: 1,979 TFLOPS FP16/BF16 and 3,958 TFLOPS FP8. The key difference is memory: 141GB at 4,800 GB/s bandwidth — 76% more capacity and 43% more bandwidth than the H100 80GB.

The 141GB HBM3e configuration fits a 70B parameter model at FP16 on a single card (140GB required) — something the H100 80GB cannot do without quantization. For 70B inference, this eliminates the need for tensor parallelism across two H100s, reducing latency and infrastructure complexity. The 4,800 GB/s bandwidth directly improves tokens-per-second for memory-bound inference: a 70B model at FP16 generates approximately 43% more tokens per second on H200 vs H100 at the same batch size.

The H200 is the right choice when running 70B+ models at FP16 on a single card, when memory bandwidth is the primary bottleneck for inference throughput, or when large context windows (>128K tokens) require more than 80GB of KV cache. It is not the right choice when compute is the bottleneck (same TFLOPS as H100), when models fit comfortably in 80GB, or when cost is the primary concern — H200 commands a 40–60% premium over H100 with no compute improvement.