1. What you need
Llama 3.3 70B has 70 billion parameters. The table below shows the minimum GPU memory required for the model weights alone, using the same formula as the VRAM calculator: multiply parameter count by bytes per parameter.
| Format | Bytes / param | Weights only (GB) |
|---|---|---|
| FP16 / BF16 | 2 | 140 GB |
| Q8 / INT8 | 1 | 70 GB |
| Q4 / INT4 | 0.5 | 35 GB |
These are weights-only figures. Real deployments need additional memory for the KV cache and runtime overhead, both of which grow with context length and batch size.
Llama 3.3 weights are distributed by Meta under the Llama 3.3 Community License. Downloading the weights requires accepting Meta's terms on the Hugging Face model page before the download is permitted.
2. Pick a GPU
The table below shows live prices for GPU instances with 48 GB or more VRAM — the minimum for running Llama 3.3 70B at Q4 quantization with headroom for the KV cache. Prices update every 15 minutes from provider APIs.
Format guidance: Q4 quantization (35 GB weights) fits a single 48 GB card with headroom for the KV cache. Q8 (70 GB weights) requires an 80 GB card or two 48 GB cards. FP16 (140 GB weights) requires multiple GPUs or a card with 192 GB or more. Two RTX 3090 or RTX 4090 cards (24 GB each) can run the Q4 model split across both cards using tensor parallelism — see the serving section below.
Cloud Provider | GPU Model | vRAM | vCPUs | RAM | Storage | On-Demand $/hr | Spot $/hr | Availability | Region | |
|---|---|---|---|---|---|---|---|---|---|---|
Thunder ComputeBEST | RTX A6000 | 48 GB | 8 | 48 GB | 256 GB | $0.35 | — | n/a | US | |
| L40S | 48 GB | 8 | 64 GB | 256 GB | $0.48 | $0.19 | n/a | Various | ||
| RTX A6000 | 48 GB | 6 | 48 GB | 256 GB | $0.48Estimate | — | n/a | US | ||
| A16 | 64 GB | 8 | 32 GB | 200 GB | $0.56 | — | n/a | EU-West | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $0.58 | — | n/a | CA/US | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $0.63 | — | n/a | EU-West | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $0.65Estimate | — | n/a | IN-West | ||
| A40 | 48 GB | 8 | 48 GB | 256 GB | $0.65 | — | n/a | IN | ||
| RTX PRO 6000 Blackwell | 96 GB | 6 | 96 GB | 150 GB | $0.66 | — | n/a | US-West | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $0.72Estimate | — | n/a | US | ||
| L40S | 48 GB | 32 | 128 GB | 960 GB | $0.77 | — | n/a | EU-West | ||
| RTX A6000 | 48 GB | 8 | 48 GB | 256 GB | $0.79 | — | n/a | EU-North | ||
| RTX A6000 | 48 GB | 8 | 48 GB | 256 GB | $0.79Estimate | — | n/a | EU-North | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $0.80 | — | n/a | IS | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $0.82 | — | n/a | SE | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $0.85 | — | n/a | Various | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $0.85 | — | n/a | EU | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $0.88 | — | n/a | EU | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $0.90Estimate | $0.33 | n/a | US/EU/APAC | ||
| L40S | 48 GB | 8 | 64 GB | 50 GB | $0.92 | — | n/a | UK | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $0.93Estimate | $0.34 | n/a | Various | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $0.95Estimate | $0.35 | n/a | EU/US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $0.95 | $0.38 | n/a | Various | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $0.95 | — | n/a | Global | ||
| A100 80GB | 80 GB | 28 | 119 GB | 400 GB | $0.98Estimate | — | n/a | EU/US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $0.98 | — | n/a | US/EU | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $0.99 | — | n/a | Global | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.01Estimate | $0.36 | n/a | US/EU/APAC | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.03Estimate | $0.38 | n/a | US | ||
| A100 80GB | 80 GB | 24 | 192 GB | 480 GB | $1.05Estimate | — | n/a | US-West | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.05 | — | n/a | EU-West | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.05 | — | n/a | US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.05 | — | n/a | EU | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.05 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.06Estimate | $0.39 | n/a | EU/US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.08Estimate | — | n/a | US | ||
| A100 80GB | 80 GB | 24 | 192 GB | 512 GB | $1.09Estimate | — | n/a | US/EU | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.09 | — | n/a | US | ||
| A100 80GB | 80 GB | 12 | 80 GB | 512 GB | $1.10Estimate | — | n/a | US | ||
| A100 80GB | 80 GB | — | — | — | $1.10 | — | n/a | |||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.10 | — | n/a | US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.10 | — | n/a | EU/US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.10 | — | n/a | US | ||
| RTX 6000 Ada | 48 GB | 20 | 64 GB | 2.0 TB | $1.13 | — | n/a | CH | ||
| A100 80GB | 80 GB | 8 | 64 GB | 240 GB | $1.15Estimate | — | n/a | US/EU/BR | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.15 | — | n/a | US/APAC | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.15 | — | n/a | EU | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.15 | — | n/a | US/EU | ||
| A100 80GB | 80 GB | 16 | 125 GB | 200 GB | $1.19Estimate | $0.42 | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.19 | — | n/a | Global | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.20 | — | n/a | EU/US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.20 | — | n/a | NL | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.25Estimate | $0.45 | n/a | Various | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.25 | — | n/a | EU | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.25 | — | n/a | IN | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.25 | — | n/a | US/EU | ||
| A100 80GB | 80 GB | 12 | 96 GB | 240 GB | $1.26Estimate | — | n/a | EU-West | ||
| A100 80GB | 80 GB | 12 | 90 GB | 500 GB | $1.28Estimate | — | n/a | US-East | ||
| A100 80GB | 80 GB | 30 | 200 GB | 512 GB | $1.29Estimate | — | n/a | us-east-1 | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.29 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.29 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.29 | — | n/a | Global | ||
| L40S | 48 GB | 32 | 128 GB | 960 GB | $1.30Estimate | — | n/a | EU-West | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.30 | — | n/a | EU/US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.30 | — | n/a | EU | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.35 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.35 | — | n/a | US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.35 | — | n/a | SE | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.38Estimate | — | n/a | EU-West | ||
| A100 80GB | 80 GB | 16 | 256 GB | 3.4 TB | $1.39Estimate | $0.49 | n/a | us-ashburn-1 | ||
| L40 | 48 GB | 8 | 64 GB | 256 GB | $1.39 | — | n/a | US | ||
| A100 80GB | 80 GB | 12 | 90 GB | 400 GB | $1.39 | — | n/a | US/IN | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.39 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.39 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.40 | — | n/a | EU-South | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.40 | — | n/a | US | ||
| RTX 6000 Ada | 48 GB | 8 | 48 GB | 256 GB | $1.41 | — | n/a | EU-North | ||
| RTX 6000 Ada | 48 GB | 8 | 48 GB | 256 GB | $1.41Estimate | — | n/a | EU-North | ||
| A100 80GB | 80 GB | 12 | 80 GB | 150 GB | $1.43 | — | n/a | UK | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.45Estimate | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.45 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.45 | — | n/a | EU | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.45 | — | n/a | UA/PL/SE | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.45Estimate | — | n/a | US-East | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.49 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.49 | $0.60 | n/a | Various | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.49 | — | n/a | EU | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.49 | — | n/a | EU | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $1.50 | — | n/a | CA/US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.50 | — | n/a | US | ||
| A100 80GB | 80 GB | 96 | 1152 GB | 8.0 TB | $1.53Estimate | $0.51 | n/a | us-east-1 | ||
| RTX PRO 6000 | 96 GB | — | — | — | $1.55 | — | n/a | |||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.55 | — | n/a | EU | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.55 | — | n/a | US/EU | ||
| RTX PRO 6000 | 96 GB | 16 | 128 GB | 512 GB | $1.56 | — | n/a | CA-Central | ||
| L40S | 48 GB | 16 | 64 GB | 400 GB | $1.57 | — | n/a | US | ||
| RTX 6000 Ada | 48 GB | 16 | 64 GB | 400 GB | $1.57 | — | n/a | US | ||
| A100 80GB | 80 GB | 24 | 240 GB | 512 GB | $1.59Estimate | $0.48 | n/a | US-East | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.59 | — | n/a | EU | ||
| A100 80GB | 80 GB | — | — | — | $1.59Estimate | — | n/a | Global | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.60 | — | n/a | EU-West | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.60 | — | n/a | EU-South | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.60 | — | n/a | NL | ||
| A100 80GB | 80 GB | 96 | 880 GB | 6.4 TB | $1.61Estimate | $0.56 | n/a | eastus | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.65 | — | n/a | IS | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.65Estimate | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.69 | — | n/a | KR | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.69 | — | n/a | SE | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.69Estimate | — | n/a | US | ||
| RTX PRO 6000 | 96 GB | 20 | 64 GB | 4.0 TB | $1.70 | — | n/a | CH | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.75 | — | n/a | US/EU/APAC | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.79 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.79 | — | n/a | EU | ||
| H100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.80 | — | n/a | Various | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.80 | — | n/a | EU | ||
| A100 80GB | 80 GB | 12 | 80 GB | 512 GB | $1.80Estimate | — | n/a | US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.83 | — | n/a | IN-North | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.85 | — | n/a | SE | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.89Estimate | $1.40 | n/a | US | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $1.90 | — | n/a | EU-West | ||
| A100 80GB | 80 GB | 12 | 80 GB | 512 GB | $1.90 | — | n/a | US | ||
| MI300X 192GB | 192 GB | 24 | 192 GB | 512 GB | $1.99 | — | n/a | US | ||
| H100 80GB | 80 GB | 16 | 128 GB | 512 GB | $1.99 | $0.79 | n/a | Various | ||
| A100 80GB | 80 GB | 12 | 90 GB | 400 GB | $1.99Estimate | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.05 | — | n/a | EU | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.10 | — | n/a | US/EU | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.10 | — | n/a | EU/US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 400 GB | $2.13Estimate | — | n/a | EU-North | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.15 | — | n/a | US/EU | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.19 | — | n/a | US | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $2.20 | $0.84 | n/a | Various | ||
| A100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.20 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.20 | — | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.20 | $0.66 | n/a | AP-Southeast | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.25 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.25 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.29 | — | n/a | IN | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.29 | — | n/a | Global | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.31Estimate | $0.90 | n/a | Various | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.35 | — | n/a | EU-West | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.35 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.35 | — | n/a | US/EU | ||
| H100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.36Estimate | $0.93 | n/a | US/EU/APAC | ||
| H100 80GB | 80 GB | 16 | 125 GB | 200 GB | $2.39Estimate | $0.99 | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.39 | $0.95 | n/a | Various | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.39 | — | n/a | Global | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $2.40 | $0.90 | n/a | US | ||
| A100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.40 | — | n/a | EU/US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.45 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.48Estimate | $0.95 | n/a | EU/US | ||
| H100 80GB | 80 GB | 26 | 200 GB | 512 GB | $2.49Estimate | — | n/a | us-east-1 | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.49 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.49 | — | n/a | US/APAC | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.49 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.49Estimate | $1.80 | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 400 GB | $2.55Estimate | — | n/a | EU/US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.59 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.59 | — | n/a | EU | ||
| H100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.60Estimate | $1.05 | n/a | Various | ||
| H100 SXM | 80 GB | — | — | — | $2.60 | — | n/a | |||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.61Estimate | $1.02 | n/a | US/EU/APAC | ||
| H100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.63Estimate | $1.05 | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.65 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.65Estimate | — | n/a | US | ||
| H100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.68Estimate | $1.07 | n/a | EU/US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.69 | — | n/a | EU | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.69 | — | n/a | EU | ||
| H100 80GB | 80 GB | 12 | 100 GB | 512 GB | $2.70Estimate | — | n/a | US | ||
| H100 80GB | 80 GB | 24 | 192 GB | 480 GB | $2.74Estimate | — | n/a | US-West | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.75 | — | n/a | US | ||
| H100 80GB | 80 GB | 24 | 240 GB | 512 GB | $2.79Estimate | $0.84 | n/a | US-East | ||
| H100 80GB | 80 GB | 24 | 192 GB | 512 GB | $2.79Estimate | — | n/a | US/EU | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.79 | — | n/a | EU | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.79Estimate | — | n/a | US | ||
| RTX A6000 | 48 GB | 12 | 96 GB | 512 GB | $2.80 | — | n/a | US-East | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.80Estimate | — | n/a | IN-West | ||
| H100 80GB | 80 GB | 16 | 128 GB | 512 GB | $2.83Estimate | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.85 | — | n/a | US/EU | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.85Estimate | — | n/a | US | ||
| H100 SXM | 80 GB | 20 | 160 GB | 512 GB | $2.85Estimate | — | n/a | US | ||
| H100 80GB | 80 GB | 16 | 120 GB | 400 GB | $2.89 | — | n/a | US/IN | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.89 | — | n/a | EU | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.89 | — | n/a | NL | ||
| A16 | 64 GB | 8 | 32 GB | 200 GB | $2.90 | $1.10 | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.90 | — | n/a | EU/US | ||
| H100 80GB | 80 GB | 16 | 128 GB | 480 GB | $2.95Estimate | — | n/a | US/EU/BR | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.95 | — | n/a | IS | ||
| H100 80GB | 80 GB | 26 | 188 GB | 480 GB | $2.98Estimate | — | n/a | EU-West | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.99 | — | n/a | KR | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $2.99 | — | n/a | SE | ||
| H100 80GB | 80 GB | 16 | 120 GB | 400 GB | $2.99Estimate | — | n/a | US | ||
| H100 80GB | 80 GB | 12 | 100 GB | 512 GB | $3.01Estimate | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.05 | — | n/a | EU | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.05 | — | n/a | EU | ||
| H100 80GB | 80 GB | 12 | 115 GB | 500 GB | $3.09Estimate | — | n/a | US-East | ||
| A40 | 48 GB | 12 | 96 GB | 512 GB | $3.10 | $1.20 | n/a | Various | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.10Estimate | — | n/a | EU-West | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.15 | — | n/a | SE | ||
| A16 | 64 GB | 8 | 32 GB | 200 GB | $3.20 | $1.25 | n/a | Various | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.20 | — | n/a | EU-West | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.20 | — | n/a | EU-South | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.20 | — | n/a | CA/US | ||
| H100 80GB | 80 GB | 12 | 100 GB | 512 GB | $3.20 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.20Estimate | — | n/a | US-East | ||
| A40 | 48 GB | 32 | 256 GB | 960 GB | $3.40Estimate | — | n/a | US/EU/BR | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.44 | — | n/a | US | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.49 | — | n/a | Global | ||
| RTX 6000 Ada | 48 GB | 12 | 96 GB | 512 GB | $3.50 | $1.30 | n/a | Various | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.50 | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.50 | $1.05 | n/a | AP-Southeast | ||
| H100 80GB | 80 GB | 20 | 256 GB | 3.4 TB | $3.55Estimate | $1.28 | n/a | us-ashburn-1 | ||
| A40 | 48 GB | 12 | 96 GB | 512 GB | $3.60 | $1.40 | n/a | US | ||
| L40 | 48 GB | 12 | 96 GB | 512 GB | $3.60 | $1.35 | n/a | Various | ||
| RTX 6000 Ada | 48 GB | 12 | 96 GB | 512 GB | $3.60 | $1.35 | n/a | EU/US | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.60 | — | n/a | EU-West | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.60 | — | n/a | US | ||
| B200 | 180 GB | 16 | 180 GB | 250 GB | $3.75 | — | n/a | US-West | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.75 | — | n/a | EU | ||
| A40 | 48 GB | 12 | 96 GB | 512 GB | $3.80 | — | n/a | us-west-1 | ||
| A16 | 64 GB | 8 | 32 GB | 200 GB | $3.80 | — | n/a | US-East | ||
| RTX 6000 Ada | 48 GB | 12 | 96 GB | 512 GB | $3.80 | $1.45 | n/a | US | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.80 | — | n/a | US/APAC | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.87 | — | n/a | US | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.89 | — | n/a | KR | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $3.89 | — | n/a | US/EU | ||
| L40 | 48 GB | 12 | 96 GB | 512 GB | $3.90 | $1.50 | n/a | US | ||
| L40S | 48 GB | 8 | 48 GB | 256 GB | $3.90 | — | n/a | US/EU | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $3.99Estimate | — | n/a | US | ||
| H100 80GB | 80 GB | 192 | 2048 GB | 3.8 TB | $4.10Estimate | $1.55 | n/a | us-east-1 | ||
| H100 80GB | 80 GB | 96 | 900 GB | 6.4 TB | $4.15Estimate | $1.64 | n/a | eastus | ||
| A40 | 48 GB | 12 | 96 GB | 512 GB | $4.20 | — | n/a | US-East | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $4.20Estimate | — | n/a | IN-West | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $4.29Estimate | — | n/a | US | ||
| H200 141GB | 141 GB | 20 | 188 GB | 400 GB | $4.39Estimate | — | n/a | EU/US | ||
| RTX 6000 Ada | 48 GB | 12 | 96 GB | 512 GB | $4.40 | — | n/a | US-East | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $4.40 | $1.76 | n/a | Various | ||
| L40S | 48 GB | 12 | 96 GB | 512 GB | $4.50 | — | n/a | EU/US | ||
| L40 | 48 GB | 12 | 96 GB | 512 GB | $4.50 | — | n/a | US-East | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $4.50Estimate | — | n/a | Various | ||
| H100 80GB | 80 GB | 24 | 192 GB | 512 GB | $4.50 | — | n/a | US/EU | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $4.50Estimate | — | n/a | US | ||
| H100 80GB | 80 GB | 20 | 160 GB | 512 GB | $4.51Estimate | — | n/a | EU-North | ||
| H200 141GB | 141 GB | 20 | 188 GB | 400 GB | $4.59Estimate | $1.89 | n/a | US | ||
| L40S | 48 GB | 12 | 96 GB | 512 GB | $4.60 | $1.75 | n/a | Various | ||
| L40S | 48 GB | 8 | 64 GB | 256 GB | $4.70 | $1.80 | n/a | US/EU/APAC | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $4.79Estimate | — | n/a | US/EU | ||
| L40S | 48 GB | 12 | 96 GB | 512 GB | $4.90 | $1.90 | n/a | US | ||
| H200 141GB | 141 GB | 26 | 240 GB | 512 GB | $4.99Estimate | — | n/a | us-east-1 | ||
| A100 80GB | 80 GB | 240 | 1920 GB | 3.0 TB | $5.07Estimate | $0.48 | n/a | us-central1 | ||
| L40S | 48 GB | 12 | 96 GB | 512 GB | $5.10 | — | n/a | US/EU | ||
| B200 180GB | 180 GB | 32 | 384 GB | 960 GB | $5.10 | — | n/a | EU-Central | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $5.20 | — | n/a | EU-South | ||
| H200 141GB | 141 GB | 26 | 240 GB | 512 GB | $5.25Estimate | — | n/a | US-East | ||
| L40S | 48 GB | 12 | 96 GB | 512 GB | $5.40 | — | n/a | us-east-1 | ||
| L40S | 48 GB | 12 | 96 GB | 512 GB | $5.80 | — | n/a | US-East | ||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $5.89Estimate | — | n/a | EU-North | ||
| B200 180GB | 180 GB | — | — | — | $6.30 | — | n/a | |||
| MI355X 288GB | 288 GB | — | — | — | $6.30 | — | n/a | |||
| A100 80GB | 80 GB | 8 | 64 GB | 256 GB | $6.80 | — | n/a | US/EU | ||
| B200 | 180 GB | 24 | 180 GB | 350 GB | $6.99 | — | n/a | US-West | ||
| B300 288GB | 288 GB | — | — | — | $8.49 | — | n/a | |||
| H200 141GB | 141 GB | 24 | 192 GB | 512 GB | $8.70 | — | n/a | EU-West | ||
| H100 80GB | 80 GB | 128 | 512 GB | 3.8 TB | $8.76 | — | n/a | EU-West | ||
| H200 141GB | 141 GB | 208 | 1872 GB | 3.0 TB | $10.60Estimate | — | n/a | us-central1 | ||
| A100 80GB | 80 GB | 28 | 200 GB | 1.0 TB | $10.80Estimate | — | n/a | US/EU/APAC | ||
| H100 80GB | 80 GB | 208 | 1872 GB | 3.0 TB | $11.06Estimate | $1.35 | n/a | us-central1 | ||
| MI300X 192GB | 192 GB | 24 | 192 GB | 512 GB | $28.50 | — | n/a | US-West | ||
| MI300X 192GB | 192 GB | 24 | 192 GB | 512 GB | $29.00 | — | n/a | Various | ||
| MI300X 192GB | 192 GB | 24 | 192 GB | 512 GB | $31.50 | — | n/a | EU/US | ||
| MI300X 192GB | 192 GB | 24 | 192 GB | 512 GB | $34.00 | — | n/a | us-east-1 | ||
| MI325X 256GB | 256 GB | 24 | 256 GB | 512 GB | $36.00 | — | n/a | Various | ||
| MI325X 256GB | 256 GB | 24 | 256 GB | 512 GB | $36.00 | — | n/a | US-West | ||
| MI325X 256GB | 256 GB | 24 | 256 GB | 512 GB | $38.50 | — | n/a | EU/US | ||
| MI325X 256GB | 256 GB | 24 | 256 GB | 512 GB | $42.00 | — | n/a | us-east-1 | ||
| B200 192GB | 192 GB | 32 | 384 GB | 960 GB | $50.00 | — | n/a | US-West | ||
| B200 192GB | 192 GB | 32 | 384 GB | 960 GB | $52.00 | — | n/a | US/EU | ||
| B200 192GB | 192 GB | 32 | 384 GB | 960 GB | $54.50 | — | n/a | us-east-1 | ||
| B200 192GB | 192 GB | 32 | 384 GB | 960 GB | $58.00 | — | n/a | US-East | ||
| GB200 NVL72 | 192 GB | 64 | 576 GB | 1.9 TB | $68.50 | — | n/a | us-east-1 | ||
| GB200 NVL72 | 192 GB | 64 | 576 GB | 1.9 TB | $72.00 | — | n/a | US-East | ||
| GB300 NVL72 | 288 GB | 64 | 768 GB | 1.9 TB | $90.00 | — | n/a | us-east-1 | ||
| GB300 NVL72 | 288 GB | 64 | 768 GB | 1.9 TB | $95.00 | — | n/a | US-East |
Showing 282 of 467 listings · Prices updated every 15 min via provider APIs · Data is indicative only
■ Cheapest ■ Mid ■ Expensive
Affiliate link — we may earn a commission.
Announced / Quote-Only — not included in pricing table
Vultr — AMD MI455X: announced, capacity available soon
Full-rack and data center scale reservation. No public per-GPU hourly rate published. Pricing requires direct sales engagement.
Affiliate disclosure: some provider rows include a "Visit" link. Compute Comparison may earn a commission if you sign up through that link, at no extra cost to you.
3. Rent and connect
The steps below apply to any GPU cloud provider. Provider-specific dashboards differ in layout but follow the same sequence.
- Create an account and add a payment method.
- Launch an instance with your chosen GPU. Select a PyTorch or CUDA base image if available — it saves installing the GPU driver stack manually.
- Copy the SSH command from the provider dashboard and connect:
ssh -i ~/.ssh/id_rsa user@<instance-ip>Verify the GPU is visible before installing anything:
nvidia-smi4. Serve the model
Two paths: vLLM for an OpenAI-compatible API server, or Ollama for a simpler single-command setup.
Path A — vLLM (OpenAI-compatible API)
Install vLLM and serve the model. You will need a Hugging Face token with access to the gated Llama 3.3 repository.
pip install vllm
vllm serve meta-llama/Llama-3.3-70B-Instruct \
--dtype bfloat16 \
--max-model-len 8192Test the server with a curl request:
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "meta-llama/Llama-3.3-70B-Instruct",
"messages": [{"role": "user", "content": "Hello"}]
}'For a dual 24 GB card setup (e.g. two RTX 3090 or RTX 4090), add the tensor-parallelism flag to split the model across both GPUs:
vllm serve meta-llama/Llama-3.3-70B-Instruct \
--dtype bfloat16 \
--tensor-parallel-size 2 \
--max-model-len 4096For production deployment detail — autoscaling, quantization options, and multi-node setups — see the vLLM deployment guide.
Path B — Ollama (simple single-command)
Ollama handles model download and quantization automatically. It defaults to Q4 quantization, which fits a single 48 GB card.
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Pull and run the model
ollama run llama3.3:70b5. What it costs
Your hourly cost is the on-demand or spot rate for the GPU you choose — see the live table in section 2 above. Multiply the hourly rate by the number of hours your instance runs to get your total compute cost.
To compare self-hosted cost against per-token API pricing for Llama 3.3 70B, see the Llama 3.3 70B model page, which shows current managed inference prices across providers. For the break-even framework — at what token volume self-hosting beats managed APIs — see the LLM API vs self-hosted cost comparison.