WaveNet: A Generative Model for Raw Audio, by DeepMind
In September 2016, researchers at Google DeepMind published WaveNet, a deep generative model that synthesises raw audio waveforms sample-by-sample using dilated causal convolutions. In evaluations on English and Mandarin speech, WaveNet reduced the gap between human speech and machine synthesis by more than 50 per cent compared with the best previous text-to-speech systems.
Background
Text-to-speech systems in 2016 fell into two main camps, and both had real problems. The first was concatenative synthesis, which works by stitching together short recordings of a real human voice. Done well, it could sound natural, but it required a large library of carefully recorded speech fragments, and anything outside that library came out sounding wrong. Voices were expensive to build and hard to modify.
The second approach was parametric synthesis, which uses a model to generate speech from a set of acoustic parameters. This was more flexible, but the output had to pass through a hand-built component called a vocoder, which converts those parameters into actual sound. Vocoders introduced a kind of processed, mechanical quality that listeners picked up on immediately. Neither approach gave you naturalness without a significant engineering cost somewhere.
What both systems had in common was that they did not work directly with audio. The waveform itself, the actual pattern of air pressure that reaches your ear, was always one step removed. Researchers had generally assumed that modelling audio at that level, one sample at a time at a rate of 16,000 or more samples per second, was not practical.
What happened
In September 2016, Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior and Koray Kavukcuoglu at Google DeepMind published WaveNet. The model generated raw audio waveforms directly, predicting each sample from all the samples that came before it.
The architecture that made this possible was dilated causal convolutions. A causal convolution is one that only looks backwards in time, so the model cannot cheat by seeing future samples when predicting the current one. The dilation part means the convolution skips across the sequence in an expanding pattern, so the model can take in a very long stretch of past audio without having to process every sample individually at every layer. The result was a large receptive field, meaning the model could hear a long window of context, at a manageable computational cost.
The team trained WaveNet on a 24.6-hour internal Google dataset of North American English speech and on a separate Mandarin Chinese dataset. They measured quality using mean opinion scores, where listeners rate naturalness on a scale of 1 to 5. For English, WaveNet scored 4.21, compared with 3.86 for the best concatenative system and 3.67 for the best parametric system. For Mandarin, it scored 4.08 against 3.82 for the concatenative baseline. The gap between WaveNet and natural human speech was more than 50 per cent smaller than the gap the previous best systems had left. The model also showed it could generate music and other non-speech audio, though the speech results were the main focus of the paper.
Why it mattered
WaveNet demonstrated that a single end-to-end neural architecture could model raw audio directly at the waveform level, bypassing the hand-engineered vocoders and concatenative pipelines that had dominated speech synthesis for decades. Its dilated causal convolution mechanism gave the model a large receptive field without the computational cost of recurrent layers, and the approach transferred beyond speech to music generation and other audio domains. The result set a new benchmark for naturalness in text-to-speech and influenced a generation of subsequent generative audio models.
People
Aaron van Den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, Koray Kavukcuoglu
Organisations
Sources
- WaveNet: A Generative Model for Raw Audio.arXiv.Primary source
- WaveNet: A Generative Model for Raw Audio.Google DeepMind.Official
Cite this page
AI Achievements. (2016). WaveNet: A Generative Model for Raw Audio, by DeepMind. Retrieved 2026-08-22, from https://achievements.ai/milestone/wavenet-a-raw-audio-generator-deepmind
@misc{achievements_wavenet_a_raw_audio_generator_deepmind,
title = {WaveNet: A Generative Model for Raw Audio, by DeepMind},
author = {{AI Achievements}},
year = {2016},
url = {https://achievements.ai/milestone/wavenet-a-raw-audio-generator-deepmind}
}