GPU Leaderboard
34 GPUs ranked by performance-per-dollar, efficiency, and raw specs
Compare AI GPUs across six dimensions: compute value (TFLOPS/$), bandwidth value (GB/s/$), power efficiency (TFLOPS/watt, GB/s/watt), and raw specs. Live cloud prices update in real time from 94+ providers. Click any GPU to see full specs and provider pricing.
FP16 TFLOPS ÷ best on-demand $/hr. Higher = more compute per dollar. Best metric for training and compute-bound workloads.
All metrics — top 15 GPUs
Sorted by FP16 TFLOPS. Click column headers to sort.
| GPU | FP16 T ↓ | BW GB/s | VRAM | TDP W | T/W | GB/s/W | T/$ | GB/s/$ |
|---|---|---|---|---|---|---|---|---|
| NVIDIA GB200 NVL72 | 9.0k | 16.0k | 3843 | 2700 | 3.33 | 5.93 | … | … |
| NVIDIA B300 192GB | 5.0k | 8.0k | 1923 | 1000 | 5.00 | 8.00 | … | … |
| NVIDIA B200 180GB | 4.5k | 8.0k | 1803 | 1000 | 4.50 | 8.00 | … | … |
| AMD Instinct MI355X | 2.6k | 8.0k | 2883 | 750 | 3.49 | 10.67 | … | … |
| NVIDIA H100 NVL | 2.0k | 3.9k | 943 | 400 | 4.95 | 9.85 | … | … |
| AMD Instinct MI300X | 1.3k | 5.3k | 1923 | 750 | 1.74 | 7.07 | … | … |
| NVIDIA H100 80GB | 989 | 3.4k | 803 | 700 | 1.41 | 4.79 | … | … |
| NVIDIA H200 141GB | 989 | 4.8k | 1413 | 700 | 1.41 | 6.86 | … | … |
| NVIDIA H100 SXM5 80GB | 989 | 3.4k | 803 | 700 | 1.41 | 4.79 | … | … |
| NVIDIA RTX 5090 32GB | 838 | 1.8k | 327 | 575 | 1.46 | 3.12 | … | … |
| NVIDIA H100 PCIe 80GB | 756 | 2.0k | 802 | 350 | 2.16 | 5.71 | … | … |
| NVIDIA RTX 5080 16GB | 551 | 960 | 167 | 360 | 1.53 | 2.67 | … | … |
| NVIDIA RTX 5070 Ti 16GB | 445 | 896 | 167 | 300 | 1.48 | 2.99 | … | … |
| AMD Instinct MI250X | 383 | 3.3k | 1282 | 560 | 0.68 | 5.85 | … | … |
| NVIDIA RTX 6000 Ada | 364 | 960 | 486 | 300 | 1.21 | 3.20 | … | … |
Methodology
FP16 TFLOPS
From manufacturer datasheets. Tensor Core peak throughput with sparsity disabled for fair comparison.
Memory bandwidth
Peak theoretical bandwidth from manufacturer specs. HBM3/HBM3e figures are bidirectional.
TDP
Thermal Design Power in watts. SXM form factors may differ from PCIe variants.
Cloud prices
Best on-demand $/hr from live API aggregation across 94+ providers. Updated in real time.
TFLOPS/$
FP16 TFLOPS ÷ best on-demand $/hr. Prices update live; rankings may shift.
GB/s/$
Memory bandwidth ÷ best on-demand $/hr. Best predictor of inference cost-efficiency.
Compare two GPUs
Side-by-side with visual bars
Cost calculator
Model total rental cost
GPU spec catalog
Full specs for all GPUs
TFLOPS/$ — the most useful GPU value metric
TFLOPS per dollar measures how much FP16 compute you get per dollar of hourly cloud rental. It is the most direct measure of value for training and inference workloads where compute throughput is the bottleneck. The H100 SXM consistently leads this metric among data center GPUs — its 989 TFLOPS FP16 at $2.49–$3.50/hr on specialist providers yields 280–400 TFLOPS/$. The RTX 4090 often outperforms data center GPUs on this metric for inference workloads: at $0.44/hr and 165 TFLOPS FP16, it delivers 375 TFLOPS/$ — competitive with the cheapest H100 configurations. The A100 80GB typically falls between the two at 200–280 TFLOPS/$ depending on provider. TFLOPS/$ is most meaningful for compute-bound workloads; memory-bandwidth-bound workloads (large-batch inference, attention-heavy models) should weight GB/s/$ more heavily.
TFLOPS/watt — efficiency for power-constrained deployments
TFLOPS per watt measures compute efficiency relative to power draw — the key metric for on-premise deployments where power is a fixed constraint, and for cloud providers that charge for power separately. NVIDIA's Hopper architecture (H100, H200) leads this metric: the H100 SXM delivers 989 TFLOPS at 700W — 1.41 TFLOPS/watt. The RTX 4090 achieves 165 TFLOPS at 450W — 0.37 TFLOPS/watt — significantly lower efficiency despite its competitive TFLOPS/$ in the cloud. The AMD MI300X is notable here: 1,307 TFLOPS FP16 at 750W yields 1.74 TFLOPS/watt, the highest efficiency among current-generation data center GPUs. For data centers with fixed power budgets (e.g., 10 MW facility), TFLOPS/watt directly determines total cluster throughput — a 20% improvement in efficiency translates to 20% more compute from the same power infrastructure.
Memory bandwidth and LLM inference throughput
For autoregressive LLM inference, memory bandwidth (GB/s) is the primary throughput constraint — not TFLOPS. During the decode phase, the GPU reads model weights from HBM on every token generation step. A model with 70B parameters at FP16 requires 140GB of weight data to be read per forward pass; at 3.35 TB/s (H100 SXM), this takes approximately 42 microseconds per token — yielding a theoretical maximum of ~24,000 tokens/sec per GPU. The AMD MI300X's 5.3 TB/s HBM3 bandwidth makes it the highest-throughput GPU for large-model inference, despite its lower TFLOPS than the H100. The GB/s/$ and GB/s/watt leaderboard tabs surface this: for inference-heavy workloads, the MI300X and H200 often rank above the H100 SXM despite higher hourly rates, because their bandwidth advantage translates directly to higher token throughput per dollar.
How to choose the right GPU for your workload
The right GPU depends on your workload's dominant constraint. For large-model training (70B+ parameters), VRAM capacity and NVLink bandwidth are the primary constraints — the H100 SXM with NVLink 4.0 (900 GB/s bidirectional) enables tensor parallelism across 8 GPUs with minimal communication overhead. For inference serving of 7B–13B models, memory bandwidth per dollar is the key metric — the RTX 4090 and A100 40GB often win here. For fine-tuning smaller models (1B–7B), TFLOPS/$ matters most — the RTX 4090 and A100 80GB are competitive. For batch inference pipelines where throughput matters more than latency, the MI300X's 192GB HBM3 and 5.3 TB/s bandwidth enable running multiple large models simultaneously without model sharding. Use the leaderboard tabs to rank GPUs by the metric that matches your bottleneck, then cross-reference with live pricing on the rent pages.
NVIDIA Hopper vs Ampere vs Ada Lovelace
The three current GPU architectures in cloud deployments each target different use cases. Hopper (H100, H200) is the dominant data center architecture: fourth-generation Tensor Cores with FP8 support, NVLink 4.0 for multi-GPU scaling, and Transformer Engine hardware acceleration for attention mechanisms. Ampere (A100, A40, A10) is the previous generation — still widely deployed and increasingly cost-effective as H100 supply grows. Ada Lovelace (RTX 4090, L40S, L4) is NVIDIA's consumer and professional visualization architecture adapted for AI: fourth-generation Tensor Cores with FP8, but PCIe interconnect rather than NVLink, and consumer-grade reliability without ECC memory on RTX variants. For production training, Hopper is the clear choice. For inference and fine-tuning where cost matters more than peak performance, Ampere and Ada Lovelace offer compelling TFLOPS/$ at lower hourly rates.