NVIDIA A100 40GB SXM vs NVIDIA GH200 96GB HBM3
A100 40GB SXM has 58% less memory than GH200 96GB HBM3 (40 GB vs 96 GB). Its memory bandwidth is 61% less (1.555 vs 4 TB/s). In the cloud, the median on-demand A100 40GB SXM price is $1.86 per GPU-hour against $5.19 for the GH200 96GB HBM3. Per gigabyte of GPU memory, that is $0.047 vs $0.054 per hour.
Specifications
Manufacturer figures as recorded by Kovara. Higher memory and bandwidth are highlighted; neither is a benchmark of your workload.
| A100 40GB SXM | GH200 96GB HBM3 | |
|---|---|---|
| Manufacturer | NVIDIA | NVIDIA |
| Architecture | Ampere | Hopper + Grace |
| Memory | 40 GB HBM2 | 96 GB HBM3 |
| Memory bandwidth | 1.555 TB/s | 4 TB/s |
| Power (TDP) | 400 W | 900 W |
| Form factor | SXM | Grace Hopper Superchip |
Cloud prices today
Public offers Kovara tracks, normalised to US dollars per GPU-hour. Prices use on-demand terms where available; spot and reserved offers are on each GPU’s price pages. Lower is highlighted.
| A100 40GB SXM | GH200 96GB HBM3 | |
|---|---|---|
| Lowest price | $1.15 | $2.29 |
| Median price | $1.86 | $5.19 |
| Highest price | $1.99 | $6.50 |
| Median per GB of memory | $0.047 | $0.054 |
| Public offers (all terms) | 5 | 5 |
| Providers | 3 | 3 |
Where to rent each
Lowest tracked price per provider. Open a provider to see every offer, region and term.
Questions
Which is cheaper to rent, the A100 40GB SXM or the GH200 96GB HBM3?
Right now the cheapest on-demand A100 40GB SXM Kovara tracks is $1.15 per GPU-hour and the cheapest GH200 96GB HBM3 is $2.29. Medians are $1.86 and $5.19. The cheaper GPU per hour is not always cheaper per job: a faster or larger GPU can finish sooner or need fewer units.
Which has more memory?
The GH200 96GB HBM3 has more memory: 96 GB against 40 GB. Memory decides which models and batch sizes fit on one GPU.
Are these benchmark results?
No. This page compares manufacturer specifications and public prices. Real performance depends on the model, precision, software and interconnect. Kova can help you estimate cost for a specific workload.