A graphics processing unit is a programmable parallel processor. NVIDIA describes CUDA as a parallel computing platform and programming model, and it is used across deep learning, scientific computing and high-performance computing, which is why the same GPU can run model training, inference and simulation. An application-specific integrated circuit, or ASIC, is designed for a narrower job. Google describes its TPUs this way: custom ASICs built to accelerate machine learning workloads, with every program compiled by the XLA compiler. [1][2]

The distinction has blurred. Modern GPUs contain specialised matrix units, and NVIDIA's Hopper generation added FP8 tensor core arithmetic that NVIDIA says doubles throughput compared with FP16 or BF16. Meanwhile, the newest ASICs include vector units, programmable kernel languages and support for mainstream frameworks. Both camps are converging on dense low-precision matrix math surrounded by as much fast memory and interconnect bandwidth as possible. [3][2][4]

It is more useful to think of a spectrum than a binary. At one end sit fully programmable GPUs; in the middle, cloud accelerators such as TPU and Trainium that run standard frameworks through a compiler; at the far end, architectures such as wafer-scale engines or dataflow inference chips that reorganise the whole system around one class of model.

The clearest early evidence came from Google's first TPU. In the 2017 ISCA paper describing it, Google reported that the chip ran production inference 15 to 30 times faster than contemporary Haswell CPUs and K80 GPUs, and delivered 30 to 80 times better performance per watt. The design removed features general-purpose processors rely on, such as caches and out-of-order execution, in favour of a deterministic execution model with a large 8-bit matrix unit. [5]

Those gains came with limits. The same paper noted that the first TPU was often held back by memory bandwidth, and Google's current guidance still says TPUs are a poor fit for workloads with frequent branching, many element-wise operations, high-precision arithmetic or custom operations inside the training loop. Specialisation helps most when a workload looks like the one the chip was designed around. [5][2]

Today's hyperscaler chips are marketed on price-performance rather than raw speed. AWS says Trn2 instances offer 30 to 40 percent better price-performance than its GPU-based P5e and P5en instances, and Microsoft says Maia 200 delivers 30 percent better performance per dollar than the latest hardware previously in its fleet. These are vendor claims, measured on workloads each vendor chose. [6][7]

Google's TPU line is the most mature. TPU7x, the first Ironwood chip, became generally available on Google Cloud in March 2026 with 192 GB of HBM per chip and pods of up to 9,216 chips. AWS offers two families: Inferentia for inference and Trainium for training and serving, with Trainium3 UltraServers of up to 144 chips available since December 2025. [8][4][9]

Microsoft announced Maia 200 in January 2026 as an inference accelerator built on a 3nm process, with 216 GB of HBM3e and over 10 petaflops of FP4 compute. Microsoft said it runs workloads including OpenAI models and Microsoft Copilot, and launched a preview SDK with PyTorch and Triton support. Meta's MTIA line serves Meta's own ranking, recommendation and generative AI workloads; in March 2026 Meta said MTIA 300 was in production and outlined MTIA 400, 450 and 500 on a cadence of roughly one new chip every six months. [7][10]

Outside the hyperscalers, Intel's Gaudi 3 pairs 128 GB of HBM2E with 24 integrated 200 Gb Ethernet ports and is available on IBM Cloud. Intel has since announced a separate inference GPU, Crescent Island, with customer sampling expected in the second half of 2026, after Gaudi adoption fell short of Intel's own targets. Cerebras builds wafer-scale processors whose latest generation packs about 4 trillion transistors and 900,000 cores on a single wafer, sold as systems or through Cerebras's own cloud. [11][12][13]

Groq, a company focused on AI inference, sells access to its technology through the GroqCloud service. In December 2025 Groq signed a non-exclusive licensing agreement with NVIDIA covering its inference technology, and its founder and president joined NVIDIA. Groq said it would remain independent under a new chief executive and that GroqCloud would keep operating. [15]

Software is where GPUs hold their largest advantage. Google recommends GPUs over TPUs for models with custom PyTorch or JAX operations that TPUs do not support, and the TPU7x generation supports only JAX and PyTorch, not TensorFlow. On AWS, custom kernels that go beyond what the Neuron compiler produces must be written in the Neuron Kernel Interface rather than CUDA. [2][8][16]

ASIC vendors are narrowing the gap by targeting the libraries most teams actually use. AWS says vLLM, Hugging Face Transformers and TorchTitan run natively on Trainium; Microsoft's Maia SDK includes PyTorch integration and a Triton compiler; Meta says MTIA builds on PyTorch, vLLM and Triton; and Intel lists PyTorch, ONNX and DeepSpeed support for Gaudi 3. [4][7][10][11]

Efficiency increasingly comes from system design as much as from the chip. Google's TPU v4 paper describes optical circuit switches that reconfigure the network between 4,096 chips at under 5 percent of system cost, and SparseCores that speed embedding-heavy models by 5 to 7 times using about 5 percent of die area. Google claims its TPU 8t and TPU 8i chips, announced in April 2026, roughly double performance per watt over Ironwood. [17][18]

Different architectures place memory differently. Cerebras compares a CS-3 system, which draws about 23 kW, with an NVIDIA DGX B200, and argues its wafer-scale design yields about 2.2 times better performance per watt. Microsoft's Maia 200 combines 272 MB of on-chip SRAM with HBM3e, and Google's TPU 8i carries 384 MB of on-chip SRAM to keep more of a model's working data close to the compute units during inference. [14][7][18]

Availability is the most practical difference for buyers. TPUs are rentable only on Google Cloud, through Compute Engine, Google Kubernetes Engine and Google's managed AI platform. Trainium and Inferentia are offered only as Amazon EC2 instances. Maia 200 was deployed first in Microsoft's US Central Azure region, and MTIA is used inside Meta's own infrastructure. [2][16][7][10]

Some alternatives are more open. Gaudi 3 is available as virtual servers and OpenShift clusters on IBM Cloud, Cerebras sells systems and a cloud service, and GroqCloud offers hosted inference. GPUs, by contrast, are sold by several vendors and rented by hyperscalers and a large number of specialised GPU clouds, so the same model can move between providers as prices and capacity change. [11][13][15]

We think the ASIC versus GPU question is better framed as platform versus platform. A custom accelerator is worth evaluating when your workload is large, stable and built on well-supported frameworks, and when you are comfortable committing to the one cloud that offers it. GPUs remain the default when your models change often, depend on custom kernels, or need to move between providers. In either case, benchmark your own model and compare cost per token or per training step, not headline petaflops. Kovara lets you compare live GPU prices and providers, which gives you the GPU baseline any ASIC quote should beat.