L40S

The NVIDIA L40S is the best-value data center GPU for inference serving in 2025, offering 48GB of GDDR6 memory and 183 TFLOPS of FP16 throughput at a significantly lower price point than the H100 or A100. Built on the Ada Lovelace architecture (AD102 die, TSMC 4N process), it was designed specifically for inference workloads — its 91.6 TFLOPS FP32 figure is unusually high for a data center GPU, making it well-suited for mixed compute and graphics workloads alongside AI inference.

The 48GB GDDR6 configuration fits models up to ~24B parameters at FP16 or ~48B at INT8. Memory bandwidth is 864 GB/s — substantially lower than HBM-based alternatives like the A100 (2,039 GB/s) or H100 (3,350 GB/s). This bandwidth gap means the L40S is less competitive for memory-bound autoregressive inference of large models, but for batch inference, image generation, and smaller model serving, the lower cost per hour often makes it the better choice. The L40S uses PCIe 4.0 with no NVLink, so VRAM cannot be pooled across cards.

The L40S is the right choice for inference serving of models up to 30B parameters, generative AI workloads (Stable Diffusion, image generation), and mixed compute environments where both AI inference and graphics rendering are needed. It is not well-suited for training large models (limited VRAM, no NVLink) or for memory-bound inference of 70B+ models where HBM bandwidth matters. At $0.80–$1.80/hr on-demand, it typically delivers better cost-per-token than A100 for models that fit in 48GB.