The H100 is the data center GPU built on NVIDIA's Hopper architecture, the successor to the Ampere-based A100. The full GH100 chip contains 80 billion transistors on an 814 square millimetre die, made on a version of TSMC's 4N process customised for NVIDIA. The full die has 144 streaming multiprocessors (SMs); shipping products enable fewer to improve yield. [2]

NVIDIA announced on September 20, 2022 that Hopper was in full production. Partners including Dell Technologies, HPE, Lenovo and Supermicro planned to roll out the first H100-based products from October 2022, with systems expected to ship in the following weeks, and AWS, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure planned H100 instances starting in 2023. [4]

H100 SXM is a module that mounts on an HGX baseboard with four or eight GPUs. It enables 132 SMs, carries 80 GB of HBM3 with 3.35 TB/s of bandwidth, can be configured up to 700 W and connects to the other GPUs on the board through 900 GB/s of NVLink. It is the version behind most large H100 training clusters. [1][2]

H100 PCIe is a full-height, full-length, dual-slot card with a passive heatsink that relies on server airflow. It enables 114 SMs and uses 80 GB of HBM2e with about 2,000 GB/s of bandwidth, at a default and maximum power of 350 W. Pairs of cards can be joined with NVLink bridges for up to 600 GB/s. The lower power limit, fewer SMs and slower memory make it substantially slower than the SXM version. [3][2]

H100 NVL, introduced in March 2023, is a PCIe card aimed at large language model inference in mainstream servers. It has 94 GB of memory with 3.9 TB/s of bandwidth, runs at 350 to 400 W, and supports 600 GB/s of NVLink between bridged cards. At launch NVIDIA claimed up to 12 times faster GPT-3 inference than the prior-generation A100 at data center scale, a vendor figure rather than a per-card comparison. [1][5]

When a cloud listing just says H100, it usually means the SXM version in an eight-GPU HGX server. PCIe and NVL instances exist but are typically labelled as such. Because the performance gap between SXM and PCIe is large, it is worth checking which one a provider is offering before comparing prices.

Hopper's fourth-generation Tensor Cores deliver twice the matrix math rate of the A100 per SM on the same data types, and four times the rate when using the new FP8 format. NVIDIA supports two FP8 variants: E4M3, with four exponent and three mantissa bits, which favours precision, and E5M2, with five exponent and two mantissa bits, which favours range. [2]

The Transformer Engine is the combination of these Tensor Cores with NVIDIA software that chooses, layer by layer, between FP8 and 16-bit precision and handles the scaling and casting between the two formats automatically. The aim is to get most of FP8's speed without the accuracy problems of applying it blindly to every layer. [2]

NVIDIA rates the H100 SXM at 3,958 teraFLOPS of FP8 and 1,979 teraFLOPS of FP16 or BF16, but these Tensor Core figures assume structured sparsity. Dense throughput is half: about 2 petaFLOPS of FP8 and about 1 petaFLOPS of BF16. For comparison, the A100 80GB is rated at 312 teraFLOPS of dense BF16. [1][8][6]

The H100 SXM uses 80 GB of HBM3 with 3.35 TB/s of bandwidth, up from 2,039 GB/s on the A100 80GB SXM. Hopper also has a 50 MB L2 cache and up to 228 KB of configurable shared memory per SM. New programming features include thread block clusters, distributed shared memory that lets SMs read each other's shared memory directly, and the Tensor Memory Accelerator for moving large blocks of data between global and shared memory. [1][2][6]

Fourth-generation NVLink gives each H100 SXM 900 GB/s of GPU-to-GPU bandwidth over 18 links, compared with 600 GB/s on the A100. Inside an HGX H100 board, NVLink 4 switches connect eight GPUs at 7.2 TB/s of aggregate bandwidth. NVIDIA also described an external NVLink Switch System able to connect up to 256 H100 GPUs, although most H100 clusters scale beyond eight GPUs over InfiniBand or Ethernet. [7][2][6]

Other Hopper additions include second-generation Multi-Instance GPU, which can split an H100 SXM into up to seven isolated instances of 10 GB each, confidential computing support, PCIe Gen5 with 128 GB/s of total bandwidth, and DPX instructions that NVIDIA says accelerate dynamic programming algorithms by up to 7 times over the A100. [1][2]

The H100 reached the major cloud providers from 2023. It arrived with FP8 and the Transformer Engine designed specifically around transformer models, a large increase in bandwidth over the A100, and NVIDIA's CUDA software stack already in place. NVIDIA and others continue to use it as the comparison point: NVIDIA's Blackwell materials, for example, express memory and performance gains relative to the H100. [4][8]

In our view the H100 became the reference point less because of any single specification than because of timing and ubiquity. It was the GPU most AI teams could actually obtain during the years when most of today's training and inference software was written and tuned, so kernels, libraries, benchmarks and cluster designs all default to it.

Two newer lines now sit above the H100. The H200 keeps the Hopper compute design but moves to 141 GB of HBM3E with 4.8 TB/s. The Blackwell B200 offers up to 192 GB of HBM3E at 8 TB/s, 1.8 TB/s NVLink and FP4 support, at up to 1,200 W. See Kovara's H100 vs H200 comparison for the memory difference in detail. [8]

As of September 2026 the H100 is no longer NVIDIA's newest GPU, but it is far from obsolete. For models that fit in 80 GB per GPU, or that shard cleanly across eight GPUs, it often remains a practical choice, particularly where its mature software support matters more than peak throughput. Its main limitation is memory: very large models or long contexts may need more GPUs on H100 than on an H200 or B200.

H100 capacity is available as single GPUs, full eight-GPU HGX nodes and larger InfiniBand clusters, on demand, as spot capacity and on reserved terms. Prices differ widely between providers and between SXM and PCIe versions, and they have moved considerably since 2023. On Kovara you can open the H100 SXM page to see current listings, check the GPU prices view to compare providers, and use Compare to weigh an H100 against an H200 or B200 for the model you plan to run.