AlexNet and Deep Convolutional Neural Networks in Large-Scale Image Classification
In September 2012, Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton at the University of Toronto submitted a paper describing AlexNet, a deep convolutional neural network that achieved a top-5 error rate of 15.3% on the ImageNet Large Scale Visual Recognition Challenge, outperforming the next-best entry by more than 10 percentage points.
Background
For most of the 2000s, the best image recognition systems were built by hand. Researchers would look at a problem like “identify a dog in a photograph” and write rules to detect edges, then textures, then shapes, assembling these into a pipeline of features they had designed themselves. The approach worked, up to a point. On clean, constrained images it could be quite reliable. On the messy variety of the real world, it hit a ceiling.
Neural networks had been around since the 1980s, and convolutional neural networks, which scan an image in overlapping patches rather than processing every pixel separately, had shown real promise for small tasks. Yann LeCun’s work on digit recognition in the 1990s was the clearest proof of that. But scaling them up required more data, and more computing power, than most groups had been able to bring to bear. The ImageNet project, led by Fei-Fei Li at Stanford, had been assembling a labelled dataset of over a million photographs across a thousand categories since 2009. The annual ILSVRC competition, short for ImageNet Large Scale Visual Recognition Challenge, gave researchers a shared problem of a scale that simply had not existed before.
By 2011, the best systems on ILSVRC were achieving top-5 error rates (the fraction of images where the correct label was not among the model’s five best guesses) in the high twenties of percent. Progress from year to year was modest. Nobody had found a way to dramatically close that gap.
What happened
Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton at the University of Toronto trained a deep convolutional neural network on the ILSVRC 2012 dataset. The network, later known as AlexNet, had eight learned layers: five convolutional and three fully connected. Training it on the full ImageNet data would have taken an impractical amount of time on a standard processor, so they spread the computation across two graphics processing units (GPUs), which were originally built for rendering video games but turned out to be well suited to the parallel arithmetic that neural network training requires.
Two other choices helped the network train more reliably. They used ReLU activations, a simple mathematical function that passes positive values through unchanged and sets negative values to zero. Earlier networks used smoother functions that caused training to slow as networks grew deeper; ReLU sidestepped much of that. They also applied dropout, a technique that randomly switches off some fraction of a layer’s connections during training, which pushed the network to learn more general features rather than memorising the training data.
The result at ILSVRC 2012 was a top-5 error rate of 15.3%. The next-best entry that year scored 26.2%. To put that another way, Krizhevsky, Sutskever and Hinton’s system made fewer errors on more than one image in ten compared with every other competitor. Human performance on the same task was estimated at around 5%, so the network had not beaten people, but it had moved further past the field of machine systems than any single year’s improvement had managed before. The paper was presented at NeurIPS in December 2012.
Why it mattered
AlexNet's victory in ILSVRC 2012 demonstrated that deep convolutional neural networks trained on GPUs could dramatically outperform hand-engineered feature pipelines on large-scale image classification, ending a long period of incremental progress in the field. The result redirected substantial research effort across computer vision and machine learning toward deep learning architectures. It also established the practice of using large labelled datasets, specifically ImageNet, as the standard benchmark for visual recognition research.
People
Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton
Organisations
University of Toronto, ImageNet ILSVRC-2012
Sources
- ImageNet Classification with Deep Convolutional Neural Networks.Advances in Neural Information Processing Systems 25 (NeurIPS 2012).Primary source
- ILSVRC 2012 Competition Results.ImageNet Large Scale Visual Recognition Challenge.Official
- ImageNet Classification with Deep Convolutional Neural Networks (ACM Communications reprint).Communications of the ACM, Volume 60, Issue 6.Secondary
Cite this page
AI Achievements. (2012). AlexNet and Deep Convolutional Neural Networks in Large-Scale Image Classification. Retrieved 2026-08-22, from https://achievements.ai/milestone/introduction-of-deep-neural-networks
@misc{achievements_introduction_of_deep_neural_networks,
title = {AlexNet and Deep Convolutional Neural Networks in Large-Scale Image Classification},
author = {{AI Achievements}},
year = {2012},
url = {https://achievements.ai/milestone/introduction-of-deep-neural-networks}
}