GB300 SXM6

The GB300 SXM6 is a Blackwell Ultra-generation NVIDIA GPU built on the GB300 Grace Blackwell architecture, manufactured on a TSMC 4NP process node with 208 billion transistors. Released in 2025, it delivers 2800 TFLOPS of FP16 throughput and 2800 TFLOPS BF16 — the two precision formats most commonly used for transformer model training and inference. FP8 precision is supported at 5600 TFLOPS — roughly 2.0× the FP16 rate — enabling near-doubled throughput for inference workloads that can tolerate reduced numerical precision with calibration. Blackwell introduces fifth-generation Tensor Cores with FP4 and FP6 support, a new NVLink Switch fabric, and a Transformer Engine capable of 20 PFLOPS of AI compute in multi-GPU configurations. At 1200W TDP, the GB300 SXM6 sits in the ultra-high-power data center tier (1200W), requiring specialized rack infrastructure with high-density power delivery.

Memory capacity is 288GB of HBM3e with 8000 GB/s bandwidth. This determines which models can run without quantization: approximately 144B parameters at FP16 (2 bytes/param), 288B at INT8 (1 byte/param), or up to 576B parameters at INT4/GGUF quantization (0.5 bytes/param). These figures are theoretical maximums — actual capacity is reduced by KV cache, framework overhead, and activation memory, typically by 10–20% for inference and 30–40% for training. The arithmetic intensity ceiling — the ratio of compute (2800 TFLOPS) to memory bandwidth (8000 GB/s) — is approximately 350 FLOP/byte. Transformer inference is typically memory-bound below this threshold, meaning the 8000 GB/s bandwidth figure is the primary determinant of tokens-per-second for most LLM serving workloads.

Multi-GPU configurations are a first-class use case for the GB300 SXM6. NVLink-C2C / NVLink 5.0 provides 1800 GB/s of bidirectional NVLink bandwidth between cards, enabling tensor-parallel inference across multiple GPUs with near-linear VRAM scaling. A two-card configuration provides 576GB of pooled VRAM — sufficient for 288B parameter models at FP16 — while a four-card setup reaches 1152GB. NVLink's low-latency, high-bandwidth fabric makes all-reduce operations in data-parallel training significantly faster than PCIe-based alternatives, which top out at ~64 GB/s bidirectional for PCIe 5.0 x16.

The primary workloads for the GB300 SXM6 are Frontier AI training, Inference at scale, Memory-bound LLM serving. Grace Blackwell Ultra superchip — ARM Grace CPU + B300 GPU connected via NVLink-C2C. 288GB HBM3e + 480GB LPDDR5X system memory. Key limitations to factor into your evaluation: Very limited cloud availability — still ramping supply; Extremely high cost — enterprise/hyperscaler pricing only; Requires specialized NVLink-C2C infrastructure. When comparing this GPU against alternatives at similar price points, the most important metrics are memory bandwidth (for inference throughput), VRAM capacity (for model size), and FP16/BF16 TFLOPS (for training speed). Raw TFLOPS figures can be misleading for inference — a GPU with lower TFLOPS but higher memory bandwidth will often outperform a higher-TFLOPS card on tokens-per-second for memory-bound autoregressive generation.

In the broader GPU market, the GB300 SXM6 is a one-year-old design that remains competitive for most workloads. CUDA compatibility is a significant advantage: the GB300 SXM6 benefits from the full NVIDIA software ecosystem including cuDNN, TensorRT, FlashAttention, and all major inference frameworks (vLLM, TGI, TensorRT-LLM). CUDA's maturity means optimized kernels are available for virtually every model architecture. For cloud rental, availability varies significantly by provider — some specialize in this GPU tier while others may have limited stock. Compare on-demand and spot pricing across providers using the rental comparison table on this page, and factor in region availability if latency is a concern for your inference workload.