H20 96GB

Specs, explained

Used for AI inference, training and China-market AI compute

1. 0196 GB of HBM3 memory

   VRAM

   96 GB

   Memory type

   HBM3

   Memory bandwidth

   4 TB/s

   HBM (High Bandwidth Memory) is stacked in layers and sits right next to the processor on the same package. That short, wide path is what gives accelerators with HBM their memory bandwidth, so they are usually chosen for work limited by how fast data reaches the cores, such as large-model inference and HPC simulation.

   VRAM is the working space on the card. Model weights, activations and datasets have to fit in it, or be split across several GPUs. Rule of thumb for the weights alone at 16-bit precision (about 2 GB per billion parameters): up to roughly 48 billion parameters, before activations and cache. This is arithmetic on the recorded VRAM, not a benchmark.

2. 02Hopper architecture

   Architecture

   Hopper

   Launch year

   2024

   Lifecycle status

   Current

   Hopper added the Transformer Engine, which mixes FP8 and 16-bit math for transformer models, and fourth-generation Tensor Cores. (General context about the architecture, not a statement about this specific card.)

   Launched in 2024. Kovara's records list it as a current-generation part.

3. 03Built to scale across GPUs

   NVLink

   Supported

   MIG

   Supported

   Form factor

   HGX SXM5 / 8-GPU platform

   TDP

   400 W

   NVLink is a direct GPU-to-GPU link. Where it is supported, GPUs in the same server can exchange data faster than over PCIe alone, which matters when one training job is spread across several GPUs.

   Multi-Instance GPU (MIG) can divide one physical GPU into isolated slices, each with its own memory and compute, so several smaller workloads can share a card.

   SXM modules are mounted directly on a server baseboard rather than slotted in as cards, the layout typically used for dense multi-GPU servers.

   TDP is the rated thermal design power: the planning figure for power delivery and cooling in a server.

4. 04What it is used for

   Primary use (recorded)

   AI inference / training / China-market AI compute

   These are the uses Kovara has recorded for this part. The notes explain the workload in general terms; they are not performance claims.

   • AI inferenceRunning a trained model to answer requests. Memory capacity, latency and cost per request are the usual concerns.
   • trainingTeaching a model from data. Usually the most compute- and memory-hungry phase, often spread across many GPUs.
   • China-market AI compute

Figures are Kovara master data, last verified 03 Sept 2026. Explanations describe the technology in general; they are not benchmarks and say nothing about price or availability.