10 June 2014

Goodfellow and colleagues propose generative adversarial networks

Ian Goodfellow and seven co-authors proposed training two neural networks against each other: one generating samples, one judging them. The setup, requiring only backpropagation and no Markov chains, could recover the training data distribution under idealised theoretical assumptions.

Learning methods Generative models Deep learning Neural networks Foundational method Demonstrated

Background

Building a model that could generate realistic data, not just classify it, was hard. Most approaches leaned on Markov chains, which sample step by step from an approximation of the target distribution, or on unrolled inference networks that tried to approximate the same thing differently. Both added complexity and made training difficult to scale.

The core problem was getting a generative model to match the true distribution of the training data. You needed something to compare the generated output against reality, and doing that directly was computationally expensive.

What happened

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio submitted a paper to arXiv on 10 June 2014 proposing a different approach. Rather than comparing generated samples against the data directly, they set two models against each other.

The first model, G, tries to capture the data distribution by generating samples. The second, D, tries to tell those generated samples apart from real training data. G’s job is to fool D. D’s job is to not be fooled. Goodfellow and colleagues described this as “a minimax two-player game,” and they showed that a unique solution exists: G recovers the training data distribution, and D outputs one-half everywhere, meaning it can no longer distinguish real from generated.

When both G and D are built as multilayer perceptrons, the whole system trains with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during training or during generation. Experiments in the paper offered qualitative and quantitative evaluation of the generated samples.

The paper appeared at the 2014 NeurIPS conference as well, under the title “Generative Adversarial Nets.”

Why it mattered

The framework offered a new way to train generative models without Markov chains or approximate inference networks, both of which had complicated earlier approaches. Under idealised assumptions of infinite capacity, the generator recovers the training distribution in theory; practical convergence is not guaranteed.

People

Yoshua Bengio

Sources

Generative Adversarial Networks. arxiv.org. Primary source
Generative Adversarial Nets. papers.nips.cc. Primary source

Cite this page

AI Achievements. (2014). Goodfellow and colleagues propose generative adversarial networks. Retrieved 2026-08-29, from https://achievements.ai/milestone/goodfellow-and-colleagues-propose-generative-adversarial

@misc{achievements_goodfellow_and_colleagues_propose_generative_adversarial,
  title  = {Goodfellow and colleagues propose generative adversarial networks},
  author = {{AI Achievements}},
  year   = {2014},
  url    = {https://achievements.ai/milestone/goodfellow-and-colleagues-propose-generative-adversarial}
}