Feed-forward neural network
A neural network where information flows in one direction only, from input to output. It is the foundation of most modern deep learning architectures.
2 milestones
A Neural Probabilistic Language Model by Yoshua Bengio and Colleagues
In 2003, Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin at the Université de Montréal published 'A Neural Probabilistic Language Model' in JMLR, demonstrating that a feed-forward neural network trained on word sequences could learn distributed word representations and outperform n-gram models on perplexity benchmarks.
NETtalk Neural Network Developed by Terrence J. Sejnowski and Charles Rosenberg
Terrence J. Sejnowski of the Salk Institute and Charles Rosenberg of Princeton University developed NETtalk, a feedforward neural network trained to convert English text to speech, publishing the principal account in Complex Systems in 1987. The network learned pronunciation from examples alone, demonstrating that a multi-layer perceptron could acquire a complex linguistic skill without hand-coded rules.