High Bandwidth Memory stacks several DRAM dies on top of one another, links them with through-silicon vias, and exposes a very wide interface to the processor sitting beside it. The first JEDEC HBM standard, JESD235, was published in October 2013, after AMD and SK hynix had worked on the technology together and brought it to the standards body. SK hynix describes itself as having released the first HBM product that same year. [1][8]

Every generation since has pulled on the same four levers: how fast each data pin runs, how many pins the stack exposes, how many DRAM dies are stacked, and how dense each die is. The JEDEC documents for HBM2, HBM3 and HBM4 show all four moving. HBM2 used a 1,024-bit interface split into 8 channels; HBM3 doubled the per-pin rate and moved to 16 channels, and its quoted 819 GB/s at 6.4 Gb/s per pin corresponds to the same 1,024-bit width; HBM4 doubled the interface to 2,048 bits and 32 channels. [3][5][6]

First-generation HBM appeared in a consumer product in 2015, when AMD launched the Radeon R9 Fury X. Each four-high stack held 1 GB and delivered 128 GB/s, and the card combined four stacks for 4 GB and 512 GB/s. The memory was fast but expensive, and the product did not change the graphics market much. [1][2]

JEDEC published the HBM2 update, JESD235A, in January 2016. It kept the 1,024-bit interface and 8 independent channels, introduced a pseudo-channel mode to improve effective bandwidth, supported 2-high, 4-high and 8-high stacks, and allowed up to 8 GB and 256 GB/s per stack. Samsung began mass-producing HBM2 in January 2016, and in April 2016 NVIDIA launched the Tesla P100 using Samsung HBM2, one of the first data-center accelerators built around stacked memory. [3][2]

HBM2E is the industry name for the extended HBM2 generation. JEDEC's JESD235B update of December 2018 raised the per-pin rate to 2.4 Gb/s, for up to 307 GB/s per stack, and added 12-high stacks with capacities of up to 24 GB. Memory makers then pushed further: in July 2020 SK hynix began mass production of HBM2E running at 3.6 Gb/s per pin, which it said delivered more than 460 GB/s from an eight-high, 16 GB stack. [4][7]

NVIDIA's A100 straddles the two versions. The 40 GB A100 uses HBM2 and is rated at 1,555 GB/s, while the later 80 GB model moved to HBM2e, reaching 1,935 GB/s in the PCIe card and 2,039 GB/s in the SXM module. That is a useful reminder that the same GPU name can hide different memory generations, with a meaningful gap in bandwidth. [13]

JEDEC published HBM3 as JESD238 in January 2022. It doubled the per-pin data rate compared with HBM2 to as much as 6.4 Gb/s, or 819 GB/s per stack, and doubled the number of independent channels from 8 to 16. It supported 4-high, 8-high and 12-high stacks with a provision for 16-high, dies from 8 Gb to 32 Gb, and capacities from 4 GB to 64 GB per stack. It also added on-die error correction and lowered signalling and core voltages. [5]

The H100 is the best-known HBM3 product. NVIDIA lists the H100 SXM with 80 GB of HBM3 at 3.35 TB/s, and the H100 NVL with 94 GB at 3.9 TB/s. AMD's Instinct MI300X also uses HBM3, arranged as eight stacks for 192 GB of capacity and a peak of 5.3 TB/s, which gave it considerably more memory per accelerator than the H100 at launch. [14][18][19]

HBM3E is an extended, faster form of HBM3 that memory makers began shipping in 2024. In February 2024 Micron announced volume production of an eight-high, 24 GB HBM3E part with pin speeds above 9.2 Gb/s and more than 1.2 TB/s per stack, and said it would appear in NVIDIA's H200. In September 2024 SK hynix began mass-producing 12-high HBM3E with 36 GB per stack at 9.6 Gb/s; to keep the stack the same height as the eight-high version it thinned each DRAM die by 40 percent. [10][8]

NVIDIA describes the H200 as the first GPU with HBM3e, carrying 141 GB at 4.8 TB/s, roughly 1.4 times the H100's bandwidth. Its Blackwell-based DGX B200 system lists 1,440 GB of HBM3e across eight GPUs with 64 TB/s of aggregate bandwidth, which works out to 180 GB and 8 TB/s per GPU. AMD's MI350X and MI355X carry 288 GB of HBM3E at 8 TB/s, with memory sourced from Micron and Samsung. [15][16][20]

JEDEC released HBM4 as JESD270-4 in April 2025. The standard doubles the interface to 2,048 bits, allows up to 8 Gb/s per pin for as much as 2 TB/s per stack, doubles the independent channels to 32, supports 4-high to 16-high stacks with 24 Gb or 32 Gb dies, and tops out at 64 GB per stack. It also lets a single controller work with either HBM3 or HBM4. [6]

Shipping parts again run well above the baseline. SK hynix said in September 2025 that its finished HBM4 design exceeded 10 Gb/s per pin. Samsung announced commercial HBM4 shipments in February 2026 at a consistent 11.7 Gb/s, or 3.3 TB/s per stack, in 12-high stacks of 24 GB to 36 GB, and in May 2026 began sampling HBM4E at 14 Gb/s and up to 3.6 TB/s per stack, with a 48 GB 12-high version. [9][11][12]

On the GPU side, NVIDIA's Vera Rubin NVL72 specifications list 288 GB of HBM4 per Rubin GPU with 19.2 TB/s of bandwidth, and the same NVIDIA page says the platform is ramping into full production. AMD has previewed its MI400 series with up to 432 GB of HBM4 and 19.6 TB/s per accelerator. [17][20]

Capacity figures deserve the same care. Sold capacity can be lower than the raw capacity of the stacks, as the H200's 141 GB and DGX B200's 180 GB per GPU suggest, and one product name can span two memory generations, as with the 40 GB and 80 GB A100. When comparing offers, the useful questions are which memory generation the exact SKU uses, how much of it is usable, and what bandwidth the vendor actually specifies.

In the rental market, memory generation is a practical proxy for what a GPU can do with large models. HBM2e-era A100s remain useful and cheap for smaller models, HBM3 H100s are the default for many teams, and HBM3E parts such as the H200, B200 and MI355X are chosen when a model or a long context needs more memory per GPU. HBM4 systems are only starting to appear, and early capacity is likely to be scarce and expensive. On Kovara you can compare hourly prices across these generations, open a GPU's page to check its exact memory type, capacity and bandwidth, or ask Kova which memory configuration fits your model.