Training large models
Training a large model means many GPUs working together on one job. The run is only as fast as its slowest link, so the network and memory matter as much as the GPU model.
What matters most
GPU-to-GPU bandwidth
Inside a server, NVLink-connected SXM GPUs communicate much faster than PCIe cards. Communication-heavy jobs benefit most.
Cluster networking
Across servers, InfiniBand or a well-built RoCE fabric keeps GPUs busy. Ask what network a cluster offer really includes.
Memory per GPU
More HBM per GPU means larger batch sizes or fewer shards. Check VRAM before comparing prices.
Contiguous capacity
Training needs many GPUs in the same place at the same time, so confirmed availability matters as much as price.
A typical setup
Recent data center GPUs (Hopper- or Blackwell-class) in SXM form factor, eight per node, connected by InfiniBand or equivalent, in one region.
How it is usually bought
Reserved or committed capacity or a dedicated cluster is common, because runs are long and need guaranteed availability. Bare metal is popular for control and consistent performance.
Common mistakes
- Comparing per-GPU prices without checking the interconnect.
- Assuming a listed price means the GPUs are free to start on your date.
- Ignoring egress and storage costs for checkpoints and datasets.
Terms to know
This is general guidance, not a recommendation for a specific provider or price. Check current offerings and confirm availability and terms with the provider.
Ready to look at real options?
See tracked offerings with their sources and dates, or ask Kova to size it for you.