H100 40GB

The H100 40GB is a PCIe-form-factor Hopper GPU that brings the Transformer Engine and FP8 support to standard server deployments. It combines 40GB of HBM3 with 1,513 TFLOPS of FP16/BF16 performance and 3,026 TFLOPS at FP8, at a 350W power target that is easier to deploy than an SXM H100.

Its 2,000 GB/s HBM3 bandwidth is strong for a PCIe accelerator and materially improves transformer inference over older Ampere cards. The trade-off is capacity: 40GB limits native FP16 placement to roughly 20B parameters before runtime overhead, KV cache, and batch size are considered, even though the compute engine is much faster than that ceiling suggests.

This model fits mid-size LLM inference, FP8-aware serving, and training jobs that favor PCIe density over an SXM platform. It is not the best choice for single-card 70B-class FP16 inference or applications whose context cache consumes most of the 40GB allocation.