GH200 Grace Hopper 96GB
Grace Hopper superchip — ARM Grace CPU + H100 GPU connected via NVLink-C2C at 900 GB/s. 96GB HBM3 + 480GB LPDDR5X system memory accessible as unified pool.
GH200 Grace Hopper 96GB Overview
The NVIDIA GH200 combines a Hopper GH100 GPU with an Arm Grace CPU in one superchip, linking them through 900 GB/s NVLink-C2C. The GPU contributes 96GB of HBM3, 1,979 TFLOPS of FP16/BF16 performance, and FP8 Transformer Engine support, while the Grace side adds large LPDDR5X system memory to the addressable platform.
The HBM3 subsystem reaches 4,000 GB/s, making GH200 especially capable for memory-bound transformer inference. Unified CPU-GPU memory access can simplify workloads whose data or context exceeds GPU memory, but CPU-attached memory is not a substitute for HBM bandwidth; performance still depends on where active tensors reside.
GH200 is tailored to large-context LLM inference, data-intensive HPC, and applications that benefit from tightly coupled CPU and GPU memory. Its specialized form factor and high platform power make it unsuitable for ordinary PCIe GPU servers or workloads that only need standard H100 compute.
Memory
Compute Performance
Hardware
Relative Performance
Relative to highest-spec GPU in database
Limitations
Live Cloud PricingOn-demand hourly rates
Compare GH200 Grace Hopper 96GB vs…
Use Case Guidance
LLM Model Size Guidance
Estimates only. Actual capacity depends on context length, KV cache, and framework overhead.
Related Guides
LLM APIs Running on This GPU Class
Providers that serve frontier LLM inference on Hopper-class hardware.
Related GPUs
Frequently Asked Questions
How much VRAM does the GH200 Grace Hopper 96GB have?
The GH200 Grace Hopper 96GB has 96GB of HBM3 memory with 4000 GB/s bandwidth. This enables running models up to approximately 192B parameters at INT4 precision, 96B at INT8, or 48B at FP16.
What is the FP16 performance of the GH200 Grace Hopper 96GB?
The GH200 Grace Hopper 96GB delivers 1979 TFLOPS of FP16 performance and 1979 TFLOPS BF16, and 3958 TFLOPS FP8. INT8 throughput is 3958 TOPS. For transformer inference, memory bandwidth (4000 GB/s) is often the binding constraint rather than raw TFLOPS.
What is the GH200 Grace Hopper 96GB best used for?
The GH200 Grace Hopper 96GB is best suited for: Memory-bound LLM inference, Large context windows, Unified CPU+GPU workloads. Grace Hopper superchip — ARM Grace CPU + H100 GPU connected via NVLink-C2C at 900 GB/s. 96GB HBM3 + 480GB LPDDR5X system memory accessible as unified pool.
What interconnect does the GH200 Grace Hopper 96GB use?
The GH200 Grace Hopper 96GB uses NVLink-C2C / NVLink 4.0 with 900 GB/s NVLink bandwidth for multi-GPU configurations. NVLink enables near-linear tensor-parallel scaling across multiple cards for models that exceed single-card VRAM.
What LLM model sizes can the GH200 Grace Hopper 96GB run?
With 96GB of HBM3, the GH200 Grace Hopper 96GB can run models up to approximately 48B parameters at FP16 (2 bytes/param), 96B at INT8 (1 byte/param), or 192B at INT4/GGUF (0.5 bytes/param). These are estimates — actual capacity depends on context length, KV cache size, and framework overhead. Longer context windows require more KV cache memory, reducing the effective model size that fits.
How does the GH200 Grace Hopper 96GB compare to the A100 for LLM inference?
The GH200 Grace Hopper 96GB has 1979 TFLOPS FP16 vs the A100 80GB's 312 TFLOPS, and 4000 GB/s memory bandwidth vs the A100's 2,039 GB/s. For memory-bound autoregressive LLM inference, bandwidth is the primary determinant of tokens-per-second. The GH200 Grace Hopper 96GB's higher bandwidth gives it a throughput advantage for large model inference.
What is the power consumption of the GH200 Grace Hopper 96GB?
The GH200 Grace Hopper 96GB has a TDP (Thermal Design Power) of 900W. This is the maximum sustained power draw under full load. For data center deployments, total rack power consumption is typically 1.2–1.5× the GPU TDP when accounting for CPU, memory, networking, and cooling overhead. At 900W, the GH200 Grace Hopper 96GB is in the high-power tier — requires specialized data center infrastructure with high-density power delivery.
Ready to rent?
Compare GH200 Grace Hopper 96GB prices across 97+ providers
Live on-demand & spot rates · monthly cost estimates · availability status