Google Announces TPU v4 Tensor Processing Unit
Google announced its fourth-generation Tensor Processing Unit (TPU v4) at Google I/O in 2021. The chip, designed specifically for large-scale machine-learning workloads, offered substantially higher performance than its predecessor and was made available to researchers via Google Cloud.
Background
Training a large neural network is, at its core, a lot of the same arithmetic done over and over: multiply two matrices, add the result to another, repeat billions of times. General-purpose processors, the kind that run operating systems and web browsers, can do this work, but they were never designed for it. Graphics processing units, originally built for rendering video games, turned out to be far better suited to the task, and through the 2010s they became the standard hardware for deep learning research.
Google had a different idea. Rather than adapting hardware built for something else, they would design a chip from scratch around the specific operations that neural networks need. The first Tensor Processing Unit, the TPU v1, went into Google’s data centres in 2015. Successive generations followed, each one pushing further in the same direction: more specialised, faster at the relevant work, and more efficient per watt. By the time the third generation, TPU v3, was in use, Google was running some of the largest training jobs in the world on its own silicon.
The question, each generation, was how much further the same approach could go. Scaling a model up means more parameters, longer training runs, and a greater need for chips to communicate quickly with each other. That last part is a genuine constraint. A single chip can only do so much; large models have to be spread across many chips, and the connections between them become a bottleneck.
What happened
In 2021 Google announced the fourth generation of its Tensor Processing Unit, the TPU v4. Where earlier generations had connected chips using conventional electrical interconnects, TPU v4 introduced optical circuit switching, meaning that connections between chips inside a pod (a large cluster of TPUs working together) could be reconfigured in software. A single TPU v4 pod contained 4,096 chips. That reconfigurability let Google reshape the communication topology to suit different workloads without physically rewiring anything.
The architecture paper, published on arXiv in April 2023, described each chip as containing two tensor cores, the processing units that carry out the matrix arithmetic at the heart of neural-network training. The paper also described hardware support for embedding lookups, a type of operation common in recommendation systems that earlier accelerators handled less efficiently.
In November 2021, Google published MLPerf benchmark results using TPU v4. MLPerf is an industry benchmark that measures how quickly a system can train a set of standard models to a defined level of accuracy. Google’s TPU v4 results covered several tasks including image classification with ResNet, object detection, language model training with BERT, and recommendation modelling with DLRM. Across those tasks, TPU v4 pods were competitive with the fastest systems submitted by other organisations that round. Google also made TPU v4 available through Google Cloud that year, which meant external researchers could run jobs on the same hardware Google used internally, at least in smaller configurations.
Why it mattered
TPU v4 represented a significant step in domain-specific hardware for AI, demonstrating that custom silicon designed around the computational patterns of neural-network training could outperform general-purpose accelerators at scale. Its deployment within Google's data centres accelerated internal research on large language models, including LaMDA and later PaLM, illustrating the tight co-evolution of hardware and model scale. Making TPU v4 pods available through Google Cloud also gave external researchers access to compute resources that had previously been exclusive to Google internally.
People
Organisations
Google, Google Cloud
Sources
- TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings.arXiv.Primary source
- Google wins MLPerf benchmarks with TPU v4.Google Cloud Blog.Official
- TPU System Architecture, Cloud TPU documentation.Google Cloud.Official
Cite this page
AI Achievements. (2021). Google Announces TPU v4 Tensor Processing Unit. Retrieved 2026-08-22, from https://achievements.ai/milestone/fourth-generation-tensor-processing-unit
@misc{achievements_fourth_generation_tensor_processing_unit,
title = {Google Announces TPU v4 Tensor Processing Unit},
author = {{AI Achievements}},
year = {2021},
url = {https://achievements.ai/milestone/fourth-generation-tensor-processing-unit}
}