The language of compute.
51 essential terms, from GPUs and networking to data centers and buying models. Clear definitions with context and links to go deeper.
Understand the architecture, commercial terms and real-world infrastructure decisions behind the compute market.
Hardware
GPU generations, accelerators, high-bandwidth memory and the components behind AI systems.
A100
NVIDIA’s Ampere-generation data center GPU, the predecessor to the H100.
Accelerator
A chip designed to speed up specific workloads such as AI. GPUs are the best-known accelerators; others include TPUs and custom AI chips.
B200
An NVIDIA data center GPU from the Blackwell generation, the successor architecture to Hopper.
FLOPS
Floating-point operations per second: a measure of raw computing throughput, often quoted in teraflops or petaflops.
GPU
A graphics processing unit: a chip built to run many calculations in parallel. GPUs are the main hardware for training and running AI models.
H100
NVIDIA’s Hopper-generation data center GPU, widely used for AI training and inference. It comes in several form factors, including SXM and PCIe.
H200
An NVIDIA Hopper-generation GPU that pairs the H100’s compute design with larger and faster memory.
HBM
High Bandwidth Memory: stacked memory placed next to the GPU chip so data can move to and from it very quickly.
Memory bandwidth
How much data a GPU can read from or write to its memory each second, usually measured in terabytes per second.
MI300X
AMD’s Instinct MI300X data center accelerator for AI and high-performance computing.
PCIe
PCI Express: the standard connection between a GPU and the rest of a server. PCIe GPUs plug in as cards.
SXM
NVIDIA’s socketed GPU form factor, mounted directly on a server board and designed for NVLink and higher power.
Tensor Core
Specialised units inside NVIDIA GPUs that perform the matrix maths at the heart of AI far faster than general-purpose cores.
VRAM
The memory on a GPU, measured in gigabytes. It holds the model weights, working data and caches while the GPU runs.
Networking
Connections between GPUs and servers, from internal links to distributed networks.
InfiniBand
A high-performance networking technology with very low latency, widely used to connect servers in AI training clusters.
NVLink
NVIDIA’s high-speed link that connects GPUs directly to each other inside a server, much faster than the standard PCIe bus.
NVSwitch
A switch chip that lets every GPU in an NVLink-connected server talk to every other GPU at full speed.
RoCE
RDMA over Converged Ethernet: a way to get InfiniBand-style direct memory transfers over Ethernet networks.
AI workloads
The language of model training, inference and memory-intensive execution.
Fine-tuning
Further training an existing model on a smaller, specific dataset to adapt it to a task.
Inference
Running a trained model to produce answers or predictions for users.
KV cache
Memory that stores intermediate results of attention while a language model generates text, so they don’t have to be recomputed.
MIG
Multi-Instance GPU: an NVIDIA feature that splits one GPU into several isolated smaller GPUs.
Precision (FP8, BF16, FP16)
The number format used for calculations. Lower precision such as FP8 uses less memory and runs faster, at some cost in numerical accuracy.
Training
Teaching a model by processing large amounts of data and adjusting its parameters. It needs many GPUs working together for days or weeks.
Data centers
Power, cooling and facilities behind large-scale infrastructure.
Colocation
Renting space, power and cooling in someone else’s data center to house your own hardware.
Data residency
The requirement that data is stored and processed in a specific country or region.
Liquid cooling
Removing heat with liquid instead of air, either at the chip or across the rack. It handles much higher power densities than air.
Megawatt (MW)
A unit of power equal to one million watts. Data center capacity is usually described in megawatts of power available.
PUE
Power Usage Effectiveness: total facility energy divided by the energy used by IT equipment. A value closer to 1 means less energy is spent on overheads like cooling.
Rack density
How much power each server rack can supply and cool, measured in kilowatts per rack.
Region
A geographic area where a provider offers capacity, such as a city, metro or country.
TDP
Thermal design power: the amount of heat a chip is designed to produce, and roughly the power it draws under load, in watts.
Buying compute
Prices, contract models and the commercial terms behind GPU capacity.
Bare metal
Renting a whole physical server with no virtualisation layer between you and the hardware.
Cluster
A group of servers connected by a fast network so they can work on one job together.
Committed term
The length of time a buyer agrees to pay for capacity, such as one, three or twelve months.
Egress
Data leaving a provider’s network, which some providers charge for per gigabyte.
GPU-hour
One GPU used for one hour. It is the standard unit for comparing compute prices.
Hyperscaler
A very large cloud provider, such as the major public clouds, operating data centers at global scale.
Neocloud
A newer cloud provider that specialises in GPU and AI infrastructure rather than general-purpose cloud services.
Node
One server in a cluster. A GPU node typically holds several GPUs, commonly eight.
On-demand
Renting compute by the hour or minute with no long commitment. You can start and stop when you like, subject to availability.
Reserved capacity
Committing to use compute for a fixed term, often months, in exchange for a lower price or guaranteed availability.
SLA
Service level agreement: a provider’s written commitment on things like uptime, with remedies if it is missed.
Spot (interruptible)
Spare capacity offered at a discount that the provider can take back at short notice.
Kovara
Additional compute terminology and infrastructure concepts.
Availability status
Whether an offering is Available, Limited or Unavailable, or unknown when the source does not say.
Compute Index
Kovara’s benchmark for compute prices and availability, calculated from independent provider contributions with a published methodology.
Confidence grade
Kovara’s A–D rating of how well a fact is supported: A highly verified, B strong evidence, C incomplete evidence, D estimate or unverified.
Fixing
An official published benchmark value from the Compute Index, recorded with the methodology version used and never silently rewritten.
Indicative price
A live estimate that has not been published as an official fixing. It is labelled as indicative so it is never mistaken for one.
Normalised price
A price converted to a common unit, such as US dollars per GPU-hour, so that different offers can be compared.
Verified Provider
A provider whose identity a person at Kovara has reviewed. The status is never granted automatically.
Put the language to work.
Go from a definition to a practical infrastructure decision.
Understand the architecture.
Explore GPU generations, memory and form factors.
Compare GPUs ↗02 / ECONOMICSRead beyond the price.
Compare publicly recorded GPU-hour rates and terms.
Explore GPU prices ↗03 / RESEARCHGo deeper into compute.
Read technical explainers, reports and practical guidance.
Browse research ↗Still have a question?
Ask Kova about an architecture or suggest a term for the glossary.