Google Brain Founded by Andrew Ng and Jeff Dean
In 2011, Andrew Ng and Jeff Dean co-founded Google Brain, an internal research group at Google dedicated to large-scale deep learning. The project demonstrated that deep neural networks trained on substantial compute could learn useful representations without labelled data, reshaping how the industry approached machine learning research.
Background
By the late 2000s, neural networks had a mixed reputation. The core ideas were decades old, and researchers had shown that deeper networks, ones with more layers between input and output, could in principle learn more complex patterns. In practice, training them was painful. The computations involved were enormous, and the hardware available to most academic groups simply was not fast enough to find out whether scale might change things.
What compute did exist was being used carefully, on relatively small models trained with labelled data, where a human had already tagged every example. Labelled datasets were expensive to produce. That kept experiments modest, and modest experiments kept expectations low.
There were exceptions. Geoffrey Hinton at the University of Toronto and others had been publishing results suggesting that depth mattered, and that better training methods helped. But running those experiments at the scale needed to be sure required resources that universities rarely had. The question of what a very large neural network, trained on very large amounts of data, might actually learn had not been answered in any systematic way.
What happened
In 2011, Andrew Ng, who was then a professor at Stanford University, joined forces with Jeff Dean and Greg Corrado, both engineers at Google, to set up what became known as Google Brain. The project sat inside Google and had access to something academic groups did not: a large number of machines that could work on a problem together.
The technical infrastructure they built to make that possible was called DistBelief, a system for spreading the training of a neural network across many processors at once. Training a big network on one machine takes a long time; DistBelief let the work run in parallel across a cluster of machines. Dean and his colleagues described the system in a paper presented at NIPS in December 2012, which showed it could train models with more than one billion parameters, a scale well beyond what earlier systems had managed.
One experiment run under this infrastructure became the public face of the project. The team fed a network ten million thumbnail images taken from YouTube, with no labels attached, no human having told it what anything was. The network was not instructed to look for any particular thing. After training, one of its internal units had learned to respond strongly to faces, and another to human bodies. A third responded to cats. The New York Times reported on the experiment in June 2012, quoting the result that had required 16,000 processors to produce. The finding mattered because it showed that a large enough network, given enough unlabelled data, would organise itself around structures that were genuinely present in the world.
The initial focus of Google Brain was understanding what scale could do, in both supervised settings (where data has labels) and unsupervised ones (where it does not). Work on applications like Google Translate came later. What the group demonstrated in its first years was something more basic: that the gap between academic deep learning research and what was actually possible had been, in part, a gap in compute.
Why it mattered
Google Brain established that industrial-scale compute applied to deep neural networks could produce qualitatively new capabilities, most visibly demonstrated when its unsupervised model learned to detect cats in YouTube frames without labelled examples. The group became a significant institutional force in deep learning research, producing work on distributed training infrastructure, notably the DistBelief and later TensorFlow systems, that influenced how both academic and commercial organisations approached large-model training. Its formation marked a shift in which technology companies, rather than universities alone, became primary sites of frontier machine learning research.
People
Andrew Y. Ng, Jeff Dean, Greg Corrado
Organisations
Google, Google Brain, Stanford University
Sources
- Large Scale Distributed Deep Networks (Dean et al., NIPS 2012).Google Research / NIPS 2012.Primary source
- How Many Computers to Identify a Cat? 16,000.The New York Times.Secondary
Cite this page
AI Achievements. (2011). Google Brain Founded by Andrew Ng and Jeff Dean. Retrieved 2026-08-22, from https://achievements.ai/milestone/google-brain-project-founded-by-andrew-ng
@misc{achievements_google_brain_project_founded_by_andrew_ng,
title = {Google Brain Founded by Andrew Ng and Jeff Dean},
author = {{AI Achievements}},
year = {2011},
url = {https://achievements.ai/milestone/google-brain-project-founded-by-andrew-ng}
}