AMD launched the Instinct MI300X on December 6, 2023. At launch, AMD named Microsoft (with its Azure ND MI300X v5 virtual machines), Oracle Cloud Infrastructure and Meta among its customers, along with server makers Dell, HPE, Lenovo and Supermicro. A sister product, the MI300A, combines GPU and CPU chiplets and powers the El Capitan supercomputer at Lawrence Livermore National Laboratory. [9]

The MI325X followed as a memory upgrade on the same CDNA 3 compute design. In June 2025 AMD introduced the MI350 series, the MI350X and MI355X, on the new CDNA 4 architecture, and said they were available through cloud providers and from server makers including Dell, HPE and Supermicro. [2][10]

Where NVIDIA's Hopper is a single large die, the MI300X is assembled from many smaller dies. Eight accelerator complex dies (XCDs) hold the compute units, and these are stacked vertically on four I/O dies that hold the system infrastructure and tie the chiplets together through AMD Infinity Fabric. Eight stacks of HBM3 sit around them on the same package. In total the MI300X has 153 billion transistors, built on a mix of TSMC 5 nm and 6 nm FinFET processes. [6][1]

Each XCD has 40 physical compute units, of which 38 are enabled to improve manufacturing yield, giving 304 compute units and 19,456 stream processors across the package. Each XCD has 4 MB of L2 cache, and a 256 MB last-level cache is shared across the device. The MI300A replaces two of the eight XCDs with three CPU chiplets (CCDs), which is how AMD builds an APU from the same parts. [6][1]

The chiplet approach lets AMD mix process nodes, use smaller dies that yield better, and reuse the same building blocks across products. The trade-off is that data has to cross die boundaries, so cache and memory placement affects performance more visibly than on a monolithic GPU.

Memory capacity has been AMD's clearest differentiator. The MI300X has 192 GB of HBM3 on an 8,192-bit interface with 5.3 TB/s of peak bandwidth. At launch AMD contrasted this with the H100's 80 GB and 3.35 TB/s. The MI325X raises capacity to 256 GB of HBM3E at 6 TB/s. The MI350X and MI355X use eight 12-high HBM3E stacks for 288 GB and 8 TB/s, and an eight-GPU MI355X platform holds about 2.3 TB of HBM3E. [1][9][2][7][3]

AMD rates the MI300X at 1.3 petaFLOPS of FP16 or BF16 and 2.6 petaFLOPS of FP8 without sparsity, doubling with sparsity, plus 81.7 teraFLOPS of FP64 vector and 163.4 teraFLOPS of FP64 matrix for scientific computing. Its typical board power is 750 W and it uses the OAM module form factor with a PCIe 5.0 x16 host link. Eight Infinity Fabric links at 128 GB/s each connect it to other GPUs. The MI325X lists the same compute figures at a typical board power of 1,000 W. [1][2]

CDNA 4 reorganises the chiplets. The MI350 series still uses eight XCDs, now built on TSMC N3P, but mounts them on two larger I/O dies on N6 instead of four smaller ones. Each XCD has 36 compute units with 32 enabled, giving 256 compute units and 185 billion transistors. Matrix cores gain the OCP microscaling formats MXFP8, MXFP6 and MXFP4, and AMD doubled the execution resources for 16-bit and smaller data types. Local data share per compute unit grows to 160 KB. [7][4]

The MI355X peaks at 10.1 petaFLOPS for MXFP4 and MXFP6, and at 10.1 petaFLOPS FP8 and 5 petaFLOPS FP16 with sparsity, at a 2,400 MHz peak clock and 1,400 W typical board power. The MI350X runs at up to 2,200 MHz and 1,000 W, is designed for passive air-cooled OAM systems, and reaches 9.2 petaFLOPS of MXFP4. AMD says air-cooled racks hold up to 64 GPUs and direct liquid-cooled racks up to 128. Both use seven Infinity Fabric links at 153 GB/s each. [4][5][10]

CDNA 4 also keeps substantial FP64 throughput: AMD lists 78.6 teraFLOPS of FP64 for the MI355X and compares it with 37 teraFLOPS for the NVIDIA B200. That makes the MI350 series relevant for mixed AI and HPC buyers, not only for language model workloads. [3][4]

ROCm is AMD's open software stack for GPU-accelerated computing. It includes the HIP runtime and programming interface, an LLVM-based compiler, math and deep learning libraries such as rocBLAS, hipBLAS, MIOpen and Composable Kernel, the RCCL collective communication library, and profiling and debugging tools. HIPIFY converts CUDA source code to HIP. The major frameworks, including PyTorch, JAX and vLLM, support ROCm. [8]

AMD has iterated quickly on software. At the MI300X launch it introduced ROCm 6 with support for FlashAttention, HIPGraph and vLLM. With the MI350 series in 2025 it released ROCm 7, which AMD says improves inference performance by up to 4 times and training by up to 3 times compared with ROCm 6.0, and it opened a developer cloud for free GPU access. [9][10]

ServeTheHome, covering the MI350 launch, noted that AMD itself acknowledges it has some way to go on software maturity even as it invests heavily. In practice, popular models served through vLLM or trained through PyTorch usually run, while custom CUDA kernels, less common libraries or very new model architectures can need porting and tuning before they perform well. [11]

On paper the MI355X is positioned against the NVIDIA B200. AMD's own comparison lists the MI355X ahead of the B200 on FP16, FP8 and especially FP6 peak throughput and on FP64, and it sets the MI355X's 288 GB and 8 TB/s against 180 GB and 7.7 TB/s for the B200. Peak figures from a vendor's own comparison should be treated with care, since delivered performance depends heavily on kernels, libraries and the model being run. [3]

The larger structural gap has been scale-up networking. MI300X and MI350 systems connect eight GPUs per node with Infinity Fabric, while NVIDIA's GB200 NVL72 places 72 GPUs in one NVLink domain. ServeTheHome pointed out that a 128-GPU liquid-cooled MI355X rack is built from sixteen eight-GPU trays, a notably different approach from the 72 GPUs in a single NVLink domain of NVL72. AMD's answer is Helios, a rack-scale system with up to 72 MI400-series GPUs and 260 TB/s of scale-up bandwidth. [11][10][6]

In our view Instinct GPUs are most compelling for memory-bound inference of large models, where more HBM per GPU reduces the number of GPUs needed per replica, and for teams already using framework-level code on PyTorch or vLLM. They are a harder fit for workloads that depend on NVIDIA-specific libraries, or for very large training jobs that benefit from a 72-GPU NVLink domain.

AMD previewed the MI400 series with 432 GB of HBM4, 19.6 TB/s of bandwidth and 40 petaFLOPS of FP4 per GPU, planned for 2026 in the Helios rack. In July 2026 AMD presented Helios and the MI455X at an event in San Francisco, and reported deals with Anthropic and Microsoft. TrendForce reported that AMD expected to ship its first Helios racks later in 2026. As of September 2026, MI300X, MI325X and MI350-series GPUs are the Instinct parts most likely to be available to rent. [10][12][13]

Fewer providers list AMD Instinct GPUs than NVIDIA GPUs, and the software images they offer differ, so it pays to check the ROCm version, driver stack and framework builds before committing. If your model is memory-bound, compare the number of GPUs you would need on an MI300X or MI355X with the number needed on an H100, H200 or B200, and compare the total cost of that configuration rather than the hourly rate per GPU. On Kovara, the MI300X 192GB page shows current listings, the GPU prices view lets you compare rates across providers, and Compare lets you place an Instinct GPU next to its NVIDIA alternatives.