Use Case

LLM Inference

LLM inference — generating tokens from a trained model — is memory-bandwidth-bound. The GPU's VRAM capacity determines which model sizes fit without quantisation, while memory bandwidth governs throughput. For managed inference without GPU provisioning, LLM API providers offer per-token billing across a wide range of models and price points. This page links the GPUs, GPU cloud providers, LLM API providers, and model pages most relevant to inference workloads.