Rent T4
Compare live on-demand and spot rental prices across 97+ cloud providers. Ultra-low TDP (70W). Extremely cheap per hour. Ideal for high-density inference racks. Widely available on GCP and AWS.
Choosing the right billing model for T4
Provision and terminate at any time. Ideal for development, short experiments, and workloads with unpredictable duration.
Instances can be reclaimed when demand spikes. Best for fault-tolerant batch jobs, training with checkpointing, and preprocessing.
Lock in a rate for 1–3 months. Right for sustained production inference or long training runs where cost predictability matters.
Best use cases
- Cost-efficient inference
- High-density serving
- Legacy workloads
T4 — Specs & Benchmarks
Performance bars, compute tiers (FP32/FP16/BF16/FP8/INT8), memory specs, LLM model size guidance, and related GPU comparisons.
T4 Rental Guide
The T4 is the most cost-effective GPU for inference serving of models up to 7B parameters. At $0.10–$0.35/hr — the lowest price of any data center GPU — it delivers strong cost-per-token for quantized 7B models, BERT-class encoders, and small generative models. For high-volume serving where cost dominates and model size fits within 16GB, the T4 is often the best choice. It is widely available on GCP (n1-standard instances with T4) and AWS (g4dn instances).
Spot instances for T4 are available at 40–70% below on-demand rates on GCP and AWS. The T4's low cost makes spot pricing particularly attractive — even at on-demand rates, it is cheaper than spot pricing on most other GPUs. For batch inference jobs, preprocessing, and embedding generation, T4 spot instances often deliver the best cost-per-output available.
The main limitations to plan around are the 16GB VRAM ceiling and 320 GB/s bandwidth. For models above 7B parameters at FP16, you will need INT8 quantization (fits up to 16B) or INT4 (fits up to 32B). The low bandwidth means autoregressive inference of larger quantized models will be slow — measure tokens-per-second for your specific model before committing to T4 at scale. For models that fit comfortably in 16GB, the T4 is hard to beat on cost.
Frequently Asked Questions
How much does it cost to rent a T4?
T4 on-demand rental prices vary by provider and region. On-demand rates typically range based on availability and provider margins — use the comparison table above to see current live rates across all providers. Spot instances are generally 40–70% cheaper than on-demand but can be interrupted. Monthly cost estimates (hourly rate × 730 hours) are shown in the table for sustained workloads.
Which cloud provider has the cheapest T4?
The cheapest T4 provider changes as providers update their pricing. The comparison table above shows live rates sorted by price, so the cheapest option is always at the top. Factors beyond headline price include region (latency to your users), availability (high/medium/low), and billing granularity (per-second vs per-hour minimums).
What can I run on a T4?
With 16GB of GDDR6, the T4 can run LLM models up to approximately 8B parameters at FP16, 16B at INT8, or 32B at INT4/GGUF quantization. Common workloads include: Cost-efficient inference, High-density serving, Legacy workloads. Ultra-low TDP (70W). Extremely cheap per hour. Ideal for high-density inference racks. Widely available on GCP and AWS.
Should I use on-demand or spot pricing for T4?
Spot instances save 40–70% vs on-demand but can be interrupted when the provider needs capacity back. Use spot for: batch inference jobs, training runs with checkpointing, preprocessing pipelines, and any workload that can tolerate interruption and restart. Use on-demand for: production inference serving, interactive workloads, and jobs that cannot be interrupted. Most providers bill per second, so short on-demand jobs are not penalized by hourly minimums.
How does the T4 compare to the H100 for cloud rental?
The H100 80GB delivers 1,979 TFLOPS FP16 with 3,350 GB/s HBM3 bandwidth, compared to the T4's 65 TFLOPS FP16 and 320 GB/s bandwidth. The H100 is significantly more expensive — typically $2.50–$5.00/hr vs lower rates for the T4. For workloads that fit within 16GB and don't require FP8 precision, the T4 often delivers better cost-per-token than the H100.
What is the memory bandwidth of the T4 and why does it matter?
The T4 has 320 GB/s of memory bandwidth. For LLM inference, memory bandwidth is often more important than raw TFLOPS — each autoregressive token generation reads the full model weight matrix from VRAM, so bandwidth directly determines tokens-per-second throughput. Higher bandwidth means faster inference for the same model at the same batch size. For batch inference (processing many requests simultaneously), compute throughput becomes more important.
Can I use the T4 for Stable Diffusion or image generation?
Yes — the T4 is capable for Stable Diffusion and image generation workloads. Image generation is primarily FP32 and FP16 compute-bound, and the T4's 8.1 TFLOPS FP32 throughput determines images-per-second. The 16GB VRAM fits SDXL (requires ~6GB) and most ControlNet pipelines. For high-throughput image generation at scale, compare cost-per-image across providers using the GPU cost calculator.