MI300X 192GB
AMD's flagship AI GPU. 192GB HBM3 — most VRAM of any single GPU at launch. Strong ROCm software stack. Widely available across major cloud providers.
MI300X 192GB Overview
AMD’s MI300X is a CDNA 3 accelerator built around 192GB of HBM3, giving it one of the largest single-accelerator memory pools available. It delivers 1,307 TFLOPS of FP16/BF16 performance and targets large language-model training and inference through the ROCm ecosystem.
Its 5,300 GB/s memory bandwidth is a defining advantage for memory-bound LLM serving, and 192GB can hold very large quantized models or extensive KV cache without immediately requiring tensor parallelism. The platform uses Infinity Fabric and PCIe rather than NVIDIA NVLink, so distributed scaling and software tuning follow a different stack.
MI300X is especially compelling for large-context, open-source model serving and memory-heavy training where its capacity can reduce GPU count. The trade-off is ROCm compatibility: teams must verify kernels, inference engines, and operational tooling rather than assuming CUDA-oriented software will transfer unchanged.
Memory
Compute Performance
Hardware
Relative Performance
Relative to highest-spec GPU in database
Limitations
Live Cloud PricingOn-demand hourly rates
Compare MI300X 192GB vs…
Use Case Guidance
LLM Model Size Guidance
Estimates only. Actual capacity depends on context length, KV cache, and framework overhead.
Related Guides
LLM APIs Running on This GPU Class
Providers that serve frontier LLM inference on CDNA 3-class hardware.
Related GPUs
Frequently Asked Questions
How much VRAM does the MI300X 192GB have?
The MI300X 192GB has 192GB of HBM3 memory with 5300 GB/s bandwidth. This enables running models up to approximately 384B parameters at INT4 precision, 192B at INT8, or 96B at FP16.
What is the FP16 performance of the MI300X 192GB?
The MI300X 192GB delivers 1307 TFLOPS of FP16 performance and 1307 TFLOPS BF16, and 2614 TFLOPS FP8. INT8 throughput is 2614 TOPS. For transformer inference, memory bandwidth (5300 GB/s) is often the binding constraint rather than raw TFLOPS.
What is the MI300X 192GB best used for?
The MI300X 192GB is best suited for: Memory-bound LLM inference, Large context windows, Open-source model serving. AMD's flagship AI GPU. 192GB HBM3 — most VRAM of any single GPU at launch. Strong ROCm software stack. Widely available across major cloud providers.
What interconnect does the MI300X 192GB use?
The MI300X 192GB uses Infinity Fabric / PCIe 5.0. Without NVLink, VRAM cannot be pooled across multiple cards — the single-card capacity is the hard ceiling for model size.
What LLM model sizes can the MI300X 192GB run?
With 192GB of HBM3, the MI300X 192GB can run models up to approximately 96B parameters at FP16 (2 bytes/param), 192B at INT8 (1 byte/param), or 384B at INT4/GGUF (0.5 bytes/param). These are estimates — actual capacity depends on context length, KV cache size, and framework overhead. Longer context windows require more KV cache memory, reducing the effective model size that fits.
How does the MI300X 192GB compare to the A100 for LLM inference?
The MI300X 192GB has 1307 TFLOPS FP16 vs the A100 80GB's 312 TFLOPS, and 5300 GB/s memory bandwidth vs the A100's 2,039 GB/s. For memory-bound autoregressive LLM inference, bandwidth is the primary determinant of tokens-per-second. The MI300X 192GB's higher bandwidth gives it a throughput advantage for large model inference.
What is the power consumption of the MI300X 192GB?
The MI300X 192GB has a TDP (Thermal Design Power) of 750W. This is the maximum sustained power draw under full load. For data center deployments, total rack power consumption is typically 1.2–1.5× the GPU TDP when accounting for CPU, memory, networking, and cooling overhead. At 750W, the MI300X 192GB is in the high-power tier — requires specialized data center infrastructure with high-density power delivery.
Ready to rent?
Compare MI300X 192GB prices across 97+ providers
Live on-demand & spot rates · monthly cost estimates · availability status