H100 80GB
Flagship data center GPU. Transformer Engine with FP8 support. SXM5 form factor for max bandwidth.
H100 80GB Overview
The NVIDIA H100 80GB is the most capable widely-available GPU for large language model training and high-throughput inference as of 2025. Built on the Hopper architecture (GH100 die, TSMC 4N process), it delivers 1,979 TFLOPS of FP16 and BF16 throughput — a 6× improvement over the A100 in raw transformer compute. The headline feature is native FP8 support via the Transformer Engine: at 3,958 TFLOPS FP8, the H100 can nearly double throughput for inference workloads that tolerate reduced precision, with automatic scaling to maintain accuracy.
Memory configuration is 80GB of HBM3 with 3,350 GB/s bandwidth — 65% higher than the A100 80GB's 2,039 GB/s. This bandwidth advantage is the primary reason H100 outperforms A100 by more than the raw TFLOPS ratio suggests for memory-bound inference: each autoregressive token generation reads the full model weight matrix, so bandwidth directly determines tokens-per-second. The SXM5 form factor connects via NVLink 4.0 at 900 GB/s bidirectional, enabling near-linear tensor-parallel scaling across 8-GPU DGX H100 nodes for 70B+ model inference.
The H100 80GB is the right choice when training models above 30B parameters, running 70B+ inference at production throughput, or working with FP8 quantization pipelines. It is overkill for inference-only workloads on models below 30B parameters — the A100 80GB or L40S typically delivers better cost-per-token at those scales. At $2.50–$5.00/hr on-demand across major providers, the H100 commands a significant premium; validate that your workload is actually compute-bound (not memory-bandwidth-bound) before choosing it over cheaper alternatives.
Memory
Compute Performance
Hardware
Relative Performance
Relative to highest-spec GPU in database
Limitations
Live Cloud PricingOn-demand hourly rates
Compare H100 80GB vs…
Use Case Guidance
LLM Model Size Guidance
Estimates only. Actual capacity depends on context length, KV cache, and framework overhead.
Related Guides
LLM APIs Running on This GPU Class
Providers that serve frontier LLM inference on Hopper-class hardware.
Related GPUs
Frequently Asked Questions
How much VRAM does the H100 80GB have?
The H100 80GB has 80GB of HBM3 memory with 3350 GB/s bandwidth. This enables running models up to approximately 160B parameters at INT4 precision, 80B at INT8, or 40B at FP16.
What is the FP16 performance of the H100 80GB?
The H100 80GB delivers 1979 TFLOPS of FP16 performance and 1979 TFLOPS BF16, and 3958 TFLOPS FP8. INT8 throughput is 3958 TOPS. For transformer inference, memory bandwidth (3350 GB/s) is often the binding constraint rather than raw TFLOPS.
What is the H100 80GB best used for?
The H100 80GB is best suited for: LLM training, Large-scale inference, Scientific HPC. Flagship data center GPU. Transformer Engine with FP8 support. SXM5 form factor for max bandwidth.
What interconnect does the H100 80GB use?
The H100 80GB uses NVLink 4.0 / PCIe 5.0 with 900 GB/s NVLink bandwidth for multi-GPU configurations. NVLink enables near-linear tensor-parallel scaling across multiple cards for models that exceed single-card VRAM.
What LLM model sizes can the H100 80GB run?
With 80GB of HBM3, the H100 80GB can run models up to approximately 40B parameters at FP16 (2 bytes/param), 80B at INT8 (1 byte/param), or 160B at INT4/GGUF (0.5 bytes/param). These are estimates — actual capacity depends on context length, KV cache size, and framework overhead. Longer context windows require more KV cache memory, reducing the effective model size that fits.
How does the H100 80GB compare to the A100 for LLM inference?
The H100 80GB has 1979 TFLOPS FP16 vs the A100 80GB's 312 TFLOPS, and 3350 GB/s memory bandwidth vs the A100's 2,039 GB/s. For memory-bound autoregressive LLM inference, bandwidth is the primary determinant of tokens-per-second. The H100 80GB's higher bandwidth gives it a throughput advantage for large model inference.
What is the power consumption of the H100 80GB?
The H100 80GB has a TDP (Thermal Design Power) of 700W. This is the maximum sustained power draw under full load. For data center deployments, total rack power consumption is typically 1.2–1.5× the GPU TDP when accounting for CPU, memory, networking, and cooling overhead. At 700W, the H100 80GB is in the high-power tier — requires specialized data center infrastructure with high-density power delivery.
Ready to rent?
Compare H100 80GB prices across 97+ providers
Live on-demand & spot rates · monthly cost estimates · availability status