GPU Use Cases
T4 use cases
The T4 is a Turing-generation data-center GPU designed for high-density inference serving. At just 70W TDP, it enables extremely dense rack deployments. Its 16 GB of GDDR6 memory and 65 TFLOPS of FP16 throughput make it the most cost-effective GPU for serving quantized models at scale.
Cost-efficient inference
The T4 is the most cost-effective GPU for inference serving of models up to 7B parameters. Its 16 GB GDDR6 fits 7B models at FP16 or 16B models at INT8. At the lowest on-demand rates of any data-center GPU, it delivers strong cost-per-token for quantized 7B models, BERT-class encoders, and small generative models. It is widely available on GCP and AWS.
High-density serving
At 70W TDP, the T4 enables extremely dense rack deployments. A single 1U server can host multiple T4 cards, making it the top choice for high-volume inference serving where power efficiency and rack density are critical. It is the most widely available low-cost GPU on Google Cloud Platform.
Batch inference and embeddings
For batch inference pipelines, embedding generation, and offline scoring workloads, the T4 delivers reliable throughput at the lowest cost of any data-center GPU. Its ECC memory and data-center reliability make it suitable for production batch workloads where consumer GPUs are not appropriate.
Legacy workloads
The T4 has broad software support across CUDA-based ML frameworks. For legacy workloads that were originally developed and optimized for T4 hardware, it remains a practical choice. Its Turing architecture lacks BF16 hardware acceleration, which limits it for newer training frameworks.