GPU Use Cases
H100 80GB use cases
The H100 80GB is NVIDIA's flagship Hopper data-center GPU. Its 80 GB of HBM3 memory, 3,350 GB/s bandwidth, and native FP8 Transformer Engine make it the reference choice for large language model training and high-throughput inference at scale.
LLM training
The H100 80GB is the standard GPU for training models above 30B parameters. Its 1,979 TFLOPS of FP16 throughput and 3,350 GB/s HBM3 bandwidth reduce wall-clock training time compared to A100-class hardware. The Transformer Engine dynamically selects FP8 or BF16 precision per layer, nearly doubling effective throughput for transformer architectures. NVLink 4.0 at 900 GB/s bidirectional enables near-linear tensor-parallel scaling across 8-GPU nodes for 70B+ model training.
LLM inference
For production inference of 70B+ models, the H100 80GB delivers roughly twice the tokens per second of an A100 80GB at similar on-demand rates, primarily because memory bandwidth limits autoregressive generation. The 80 GB capacity fits a 70B model at FP16 on a single card. FP8 inference via TensorRT-LLM or vLLM can further increase throughput for latency-sensitive serving.
Scientific HPC
The H100 supports FP64 double-precision compute for scientific simulation, molecular dynamics, and climate modeling. Its NVLink fabric enables tight multi-GPU coupling for MPI workloads. For HPC jobs that can use FP32 or lower precision, the H100's throughput advantage over V100 and A100 is substantial.
Fine-tuning
Full fine-tuning of 7B–70B models benefits from the H100's bandwidth and FP8 support. LoRA and QLoRA fine-tuning of 70B models fits on a single H100 80GB with 4-bit quantization. For full-parameter fine-tuning of 70B models, two or more H100s with tensor parallelism are typically required.