A100 80GB
Workhorse of the AI era. Widely available, mature software support, excellent price/performance.
A100 80GB Overview
The NVIDIA A100 80GB is the most widely deployed GPU for production AI workloads and remains the benchmark against which newer GPUs are measured. Built on the Ampere architecture (GA100 die, TSMC 7nm), it delivers 312 TFLOPS of FP16/BF16 throughput with 80GB of HBM2e memory at 2,039 GB/s bandwidth. Released in 2020, it has the most mature software ecosystem of any data center GPU — virtually every ML framework, inference server, and cloud provider has been optimized for A100 performance.
The 80GB HBM2e configuration fits a 40B parameter model at FP16 or an 80B model at INT8. NVLink 3.0 at 600 GB/s bidirectional enables efficient multi-GPU tensor parallelism for 70B+ models across 8-GPU DGX A100 nodes. The A100 lacks native FP8 support (introduced in H100's Hopper architecture), which limits peak throughput for inference workloads that could benefit from reduced precision. INT8 inference via TensorRT is well-supported and delivers 624 TOPS.
The A100 80GB is the right choice for teams that need proven reliability, broad software support, and predictable performance. It is the most cost-effective option for training 7B–30B models and for inference serving of models that fit in 80GB. For workloads requiring FP8 throughput or models above 70B parameters, the H100 is the better choice — but at a significant cost premium. The A100 remains the most available high-end GPU across cloud providers, making it the default choice when H100 availability is limited.
Memory
Compute Performance
Hardware
Relative Performance
Relative to highest-spec GPU in database
Limitations
Live Cloud PricingOn-demand hourly rates
Compare A100 80GB vs…
Use Case Guidance
LLM Model Size Guidance
Estimates only. Actual capacity depends on context length, KV cache, and framework overhead.
Related Guides
LLM APIs Running on This GPU Class
Providers that serve frontier LLM inference on Ampere-class hardware.
Related GPUs
Frequently Asked Questions
How much VRAM does the A100 80GB have?
The A100 80GB has 80GB of HBM2e memory with 2000 GB/s bandwidth. This enables running models up to approximately 160B parameters at INT4 precision, 80B at INT8, or 40B at FP16.
What is the FP16 performance of the A100 80GB?
The A100 80GB delivers 312 TFLOPS of FP16 performance and 312 TFLOPS BF16. INT8 throughput is 624 TOPS. For transformer inference, memory bandwidth (2000 GB/s) is often the binding constraint rather than raw TFLOPS.
What is the A100 80GB best used for?
The A100 80GB is best suited for: ML training, Large model inference, HPC workloads. Workhorse of the AI era. Widely available, mature software support, excellent price/performance.
What interconnect does the A100 80GB use?
The A100 80GB uses NVLink 3.0 / PCIe 4.0 with 600 GB/s NVLink bandwidth for multi-GPU configurations. NVLink enables near-linear tensor-parallel scaling across multiple cards for models that exceed single-card VRAM.
What LLM model sizes can the A100 80GB run?
With 80GB of HBM2e, the A100 80GB can run models up to approximately 40B parameters at FP16 (2 bytes/param), 80B at INT8 (1 byte/param), or 160B at INT4/GGUF (0.5 bytes/param). These are estimates — actual capacity depends on context length, KV cache size, and framework overhead. Longer context windows require more KV cache memory, reducing the effective model size that fits.
How does the A100 80GB compare to the A100 for LLM inference?
The A100 80GB has 312 TFLOPS FP16 vs the A100 80GB's 312 TFLOPS, and 2000 GB/s memory bandwidth vs the A100's 2,039 GB/s. For memory-bound autoregressive LLM inference, bandwidth is the primary determinant of tokens-per-second. The A100's higher bandwidth gives it a throughput advantage for large model inference, despite the A100 80GB's lower cost.
What is the power consumption of the A100 80GB?
The A100 80GB has a TDP (Thermal Design Power) of 400W. This is the maximum sustained power draw under full load. For data center deployments, total rack power consumption is typically 1.2–1.5× the GPU TDP when accounting for CPU, memory, networking, and cooling overhead. At 400W, the A100 80GB is in the mid-range tier — compatible with standard data center power infrastructure.
Ready to rent?
Compare A100 80GB prices across 97+ providers
Live on-demand & spot rates · monthly cost estimates · availability status