GPU Use Cases
MI300X use cases
The AMD MI300X is a CDNA 3 data-center GPU with 192 GB of HBM3 memory and 5,300 GB/s bandwidth — the highest memory capacity and bandwidth of any widely-available GPU. It is purpose-built for memory-bound LLM inference and large context window workloads.
Memory-bound LLM inference
The MI300X's 192 GB HBM3 and 5,300 GB/s bandwidth make it the top choice for memory-bound LLM inference. Its capacity fits a 70B model at FP16 with substantial headroom for KV cache and long context windows. For inference of 70B+ models where memory bandwidth is the bottleneck, the MI300X delivers higher tokens per second than the H100 80GB. It requires ROCm-compatible inference frameworks such as vLLM with ROCm support.
Large context window workloads
The MI300X's 192 GB capacity enables extremely long context windows for LLM inference. For workloads requiring context lengths of 128K tokens or more, the MI300X's memory capacity is a decisive advantage over GPUs with 80 GB or less. This makes it well-suited for document processing, long-form generation, and retrieval-augmented generation pipelines.
Large model training
The MI300X's 1,307 TFLOPS of FP16/BF16 throughput and 192 GB capacity make it capable for training large models. Its Infinity Fabric interconnect enables multi-GPU tensor parallelism. ROCm software maturity lags CUDA for some training frameworks, so compatibility should be validated before committing to MI300X for training workloads.
Open-source model serving
The MI300X is increasingly supported by open-source inference frameworks including vLLM, TGI, and Ollama via ROCm. For teams serving open-source models such as Llama, Mistral, and Qwen, the MI300X's memory capacity enables single-card serving of models that require multiple H100s.