DeepMind's DQN learns to play Atari games from raw pixels

DeepMind's Deep Q-Networks algorithm learned to play Atari 2600 games directly from raw pixels, matching or exceeding the score of a human tester on roughly half of the games tested, without any prior knowledge of the rules.

Background

Reinforcement learning, the idea of training an agent by giving it rewards or punishments rather than labelled examples, had been studied for decades. The appeal was obvious: if an agent could learn just from trial and error, you would not need a human to label every training example. The trouble was that combining reinforcement learning with neural networks had proven unstable. Earlier attempts tended to fall apart as training progressed.

At the same time, deep learning had started producing strong results on tasks like image recognition, where large convolutional networks, ones that scan images in overlapping patches, could learn useful features directly from pixels. The open question was whether the same raw-input approach could work when the network also had to make decisions and cope with delayed rewards.

What happened

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, and colleagues at DeepMind built a convolutional neural network trained with a variant of Q-learning, a method where the network learns to estimate the total future reward expected from each possible action. The input was nothing but raw pixels from the screen; the output was a value for each joystick direction the agent could take.

To address the instability that had dogged earlier work, the Deep Q-Networks algorithm, known as DQN, stores all of the agent’s experiences and then randomly samples and replays these experiences to provide diverse and decorrelated training data. That replay mechanism broke the correlations between consecutive frames that had previously caused learning to collapse.

The team applied DQN to 50 different Atari 2600 games without adjusting the architecture or the learning algorithm between games, and without giving it any prior knowledge of the rules. DQN matched or exceeded the score of a human tester on roughly half of the games, with large variance across titles. Later improvements, including prioritising replayed experiences and normalising and aggregating the network outputs, pushed that further: combining several of those changes roughly tripled mean score across Atari games, though gains were unevenly distributed, and human-level performance was eventually reached in almost all of them.

The earlier arXiv preprint, submitted in December 2013, had already shown the approach working on seven games, outperforming all previous methods on six and surpassing a human expert on three. The full Nature paper extended that evaluation considerably and brought the result to wide attention.

Why it mattered

It was the first widely successful algorithm for deep reinforcement learning, showing that a single agent architecture could learn many different tasks from raw sensory input alone. The result showed that deep neural networks and reinforcement learning could be combined stably on high-dimensional pixel input across diverse tasks, extending beyond prior successes in more constrained domains.

People

Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou

Organisations

Deepmind

Sources

• Paper of record. doi.org. Primary source
• Playing Atari with Deep Reinforcement Learning. arxiv.org. Primary source
• Deep Reinforcement Learning, Google DeepMind. deepmind.google. Official

Cite this page

AI Achievements. (2015). DeepMind's DQN learns to play Atari games from raw pixels. Retrieved 2026-08-29, from https://achievements.ai/milestone/deepminds-dqn-learns-to-play-atari-games-from-raw-pixels

@misc{achievements_deepminds_dqn_learns_to_play_atari_games_from_raw_pixels,
  title  = {DeepMind's DQN learns to play Atari games from raw pixels},
  author = {{AI Achievements}},
  year   = {2015},
  url    = {https://achievements.ai/milestone/deepminds-dqn-learns-to-play-atari-games-from-raw-pixels}
}