A Tensor Processing Unit is a custom chip Google designed to speed up machine learning. Google's documentation describes TPUs as application-specific integrated circuits, meaning the silicon is specialised for one class of work rather than being a general-purpose processor. Every program that runs on a TPU is compiled by XLA, a just-in-time compiler that breaks large matrix multiplications into tiles sized for the hardware. [3]

Google recommends TPUs for models dominated by matrix computations, for models that train for weeks or months, and for recommendation systems with very large embedding tables. The same guidance lists cases where TPUs are a poor fit: code with frequent branching, workloads made mostly of element-wise operations, anything that needs high-precision arithmetic, and training loops that rely on custom operations. For those, Google points users to GPUs or CPUs. [3]

A TPU is best understood as a trade. By giving up flexibility that a GPU keeps, the chip can spend more of its area and power on multiply-accumulate units and on-chip memory. Whether that trade pays off depends almost entirely on how closely a workload matches what the chip was built to do.

The core of a TPU is the matrix multiply unit, or MXU, which is built as a systolic array. In a systolic array, a grid of simple arithmetic cells passes operands to its neighbours in lockstep, so each value read from memory is reused many times as it flows across the grid. Google's first-generation TPU used a 256 by 256 array of 8-bit integer multipliers, 65,536 in total, and the company's description of the chip explains that this short, neighbour-to-neighbour wiring saves both energy and memory traffic. [2][1]

Modern Cloud TPUs organise compute into TensorCores. Each TensorCore contains MXUs, a vector unit and a scalar unit. Google documents 128 by 128 MXUs on earlier Cloud TPU generations and 256 by 256 MXUs on v6e (Trillium) and TPU7x (Ironwood). By default the MXUs multiply in bfloat16 and accumulate results in FP32. [4][5]

Some TPUs also include SparseCores, dataflow processors aimed at the irregular memory lookups of embedding-heavy recommendation models. According to Google Cloud documentation, v5p and TPU7x chips each carry four SparseCores and v6e chips carry two. The TPU v4 paper reported that SparseCores sped up embedding-heavy models by roughly 5 to 7 times while using about 5 percent of die area and power. [4][7]

Google's 2017 ISCA paper describes the first TPU, which had been running inference in Google data centers since 2015. It offered a peak of 92 trillion 8-bit operations per second, carried 28 MiB of on-chip memory and was attached to host servers over PCIe. Compared with the Intel Haswell CPUs and NVIDIA K80 GPUs in the same data centers, the authors measured roughly 15 to 30 times higher performance on production neural networks and 30 to 80 times better performance per watt. [1]

The first TPU was inference-only and had deliberately simple control logic: no caches, branch prediction or out-of-order execution. Google's own write-up says the design prioritised predictable latency, which mattered for user-facing services such as search and translation. [2][1]

Later generations were offered to cloud customers for training as well as serving. Google Cloud documents TPU v3 as having two TensorCores per chip, 32 GiB of HBM2 and 123 teraflops of bfloat16 compute per chip, with 1,024 chips in a pod joined in a 2D torus. The TPU v4 paper, published in 2023, describes a 4,096-chip supercomputer deployed since 2020 that used optical circuit switches to reconfigure its interconnect, which the authors said accounted for under 5 percent of system cost. [6][7]

The v5 generation split into two products. TPU v5e is a cost-oriented chip with one TensorCore, 16 GB of HBM and 197 teraflops of bfloat16 compute, deployed in 256-chip pods on a 2D torus. TPU v5p is the high-end part: two TensorCores and four SparseCores per chip, 95 GiB of HBM at about 2.8 TB per second, and 8,960 chips per pod connected in a 3D torus, with single jobs of up to 6,144 chips. [8][9]

TPU v6e, branded Trillium, became generally available in December 2024. Each chip has one TensorCore with 256 by 256 MXUs, 32 GB of HBM and 918 teraflops of bfloat16 compute, and a full pod holds 256 chips. Google's release notes credit Trillium with more than 4 times the training performance and up to 3 times the inference throughput of the v5 generation. [10][12]

TPU7x is the first product in the seventh-generation Ironwood family. It entered preview on 24 November 2025 and became generally available on 31 March 2026. Each chip combines two chiplets for a total of two TensorCores, four SparseCores and 192 GB of HBM with about 7.4 TB per second of bandwidth. Google lists peak compute of 2,307 teraflops in bfloat16 and 4,614 teraflops in FP8, and a full pod of 9,216 chips. [12][11]

On 22 April 2026 Google announced its eighth TPU generation as two separate chips. TPU 8t targets large-scale pre-training: it keeps SparseCores and a 3D torus, scales to 9,600 chips in a superpod, carries 216 GB of HBM per chip and adds native FP4 arithmetic. TPU 8i targets serving and reasoning workloads: it carries 288 GB of HBM and 384 MB of on-chip SRAM per chip, adds a Collectives Acceleration Engine, and uses a new topology Google calls Boardfly with up to 1,152 chips per pod. [13][14]

Google claims TPU 8t improves training price-performance by 2.7 times over Ironwood and that TPU 8i improves inference price-performance by 80 percent, with both delivering about twice the performance per watt. These are vendor figures from the launch material. At announcement, Google invited customers to register interest and said the chips would be available later in 2026. General availability was not confirmed in the sources reviewed for this article as of September 2026. [13][14]

Splitting one generation into training and inference chips is a meaningful signal. It suggests that at hyperscale, the memory and networking needs of serving large models with long contexts have diverged enough from pre-training to justify separate silicon.

What sets TPUs apart from a rack of accelerator cards is the dedicated network between chips. Google defines a pod as a contiguous set of TPUs joined by a specialised network, and a slice as the subset of a pod you actually allocate, linked by high-speed inter-chip interconnect (ICI). Slices are described by 2D or 3D topologies, and Multislice connects several slices over the ordinary data-center network when a job needs more chips than one slice provides. [4]

ICI bandwidth has grown with each generation. Google documents 400 GB per second per chip on v5e, 800 GB per second on v6e and 1,200 GB per second bidirectional on both v5p and TPU7x. On TPU7x, slice topologies range from a single 4-chip host at 2x2x1 up to 8x16x16, or 2,048 chips, and each virtual machine exposes 4 chips. [8][10][9][11]

TPUs are available only through Google Cloud. Google lists Compute Engine, Google Kubernetes Engine and its managed AI platform as the access paths, and notes that the older Cloud TPU API is deprecated. Framework support covers JAX, PyTorch and TensorFlow in general, although TPU7x drops TensorFlow and supports only JAX and PyTorch. [3][11]

Google offers several consumption models. On-demand capacity is pay-as-you-go without preemption. Spot VMs are cheaper but can be reclaimed at any time. Flex-start lets a job obtain capacity for up to 7 days without a prior reservation, and calendar-mode reservations secure capacity for 1 to 90 days from a chosen start date; both apply to TPU7x, v6e and v5p. Longer reservations of a year or more come with committed-use discounts. [15]

TPUs are a strong option for teams already on Google Cloud whose models are large, matrix-heavy and written for JAX or PyTorch with XLA in mind. They are a weaker fit for code that depends on hand-written CUDA kernels or for buyers who want to move workloads freely between clouds and neoclouds. We suggest treating TPUs as one line in a broader comparison: use Kovara to see current GPU prices and provider options, then weigh them against a TPU quote for the same job, measuring cost per trained token or per served request rather than cost per chip-hour.