Best LLM Models for the NVIDIA H100 80GB
The gold standard for frontier LLM inference and fine-tuning
The H100 80GB is the most widely deployed GPU for frontier LLM inference. Its 80 GB HBM3 and 3,958 GB/s bandwidth comfortably fit Llama 3.1 405B in FP8, DeepSeek R1 671B with tensor parallelism, and every frontier closed-weight model served via API. For fine-tuning, a single H100 handles 70B models in full precision; two H100s cover 405B in LoRA. Cloud spot pricing starts around $2/hr, making it the most cost-effective frontier GPU for production inference.
Recommended Models (80GB Hopper)
Common Use Cases
VRAM Sizing Guide for NVIDIA H100 80GB
BF16 (full precision): requires ~2 bytes per parameter. On 80GB VRAM, the maximum model size in BF16 is approximately 40960B parameters (leaving ~10% headroom for KV cache).
INT8 quantisation: halves memory to ~1 byte/param — fits up to ~81920B parameters. Minimal quality loss on most models.
INT4 / Q4 quantisation: ~0.5 bytes/param — fits up to ~163840B parameters. Recommended for 70B+ models on this GPU. Use GPTQ or AWQ for best quality.
KV cache: grows with context length and batch size. At 128K context with batch size 8, a 70B model in INT4 needs ~12GB additional VRAM for KV cache. Factor this into your serving configuration.