GPU × LLMAda Lovelace · 48GB VRAM

Best LLM Models for the NVIDIA L40S

Best price-performance for 34B–70B models in cloud deployments

The L40S 48GB sits in the sweet spot between consumer and data-centre GPUs. At $1–1.50/hr spot, it runs Llama 3.3 70B in INT4 or 34B models in BF16 with excellent throughput. Its Ada Lovelace architecture brings FP8 tensor cores and improved power efficiency over the A100. For teams serving efficient-tier models at scale — Llama 3.3 70B, Mistral Large, Command R+ — the L40S delivers the best tokens-per-dollar of any data-centre GPU currently available.

VRAM fit key: BF16 fit — runs in full precision INT4 fit — runs quantised API only — closed-weight, served via API

Recommended Models (48GB Ada Lovelace)

Common Use Cases

70B INT4 inferencecost-optimised servingbatch workloadsfine-tuning 7B–13B

VRAM Sizing Guide for NVIDIA L40S

BF16 (full precision): requires ~2 bytes per parameter. On 48GB VRAM, the maximum model size in BF16 is approximately 24576B parameters (leaving ~10% headroom for KV cache).

INT8 quantisation: halves memory to ~1 byte/param — fits up to ~49152B parameters. Minimal quality loss on most models.

INT4 / Q4 quantisation: ~0.5 bytes/param — fits up to ~98304B parameters. Recommended for 70B+ models on this GPU. Use GPTQ or AWQ for best quality.

KV cache: grows with context length and batch size. At 128K context with batch size 8, a 70B model in INT4 needs ~12GB additional VRAM for KV cache. Factor this into your serving configuration.