Best LLM Models for the NVIDIA A100 80GB
Proven workhorse for 70B models and multi-GPU 405B deployments
The A100 80GB remains the most widely available high-end GPU for LLM inference, with spot pricing as low as $1.50/hr on major clouds. It comfortably runs Llama 3.3 70B in BF16, DeepSeek V3 in INT4 (with 2×A100), and all advanced-tier API models. While the H100 outperforms it on raw throughput, the A100's availability and pricing make it the practical choice for teams scaling from prototype to production. Two A100s handle 405B in INT8; four cover BF16.
Recommended Models (80GB Ampere)
Common Use Cases
VRAM Sizing Guide for NVIDIA A100 80GB
BF16 (full precision): requires ~2 bytes per parameter. On 80GB VRAM, the maximum model size in BF16 is approximately 40960B parameters (leaving ~10% headroom for KV cache).
INT8 quantisation: halves memory to ~1 byte/param — fits up to ~81920B parameters. Minimal quality loss on most models.
INT4 / Q4 quantisation: ~0.5 bytes/param — fits up to ~163840B parameters. Recommended for 70B+ models on this GPU. Use GPTQ or AWQ for best quality.
KV cache: grows with context length and batch size. At 128K context with batch size 8, a 70B model in INT4 needs ~12GB additional VRAM for KV cache. Factor this into your serving configuration.