long short-term memory
A recurrent network design with input, output, and forget gates that learns long-range dependencies in sequential data, addressing the vanishing gradient problem.
3 milestones
Andrew M. Dai and Quoc V. Le Introduced Semi-Supervised Sequence Learning
In November 2015, Andrew M. Dai and Quoc V. Le at Google Brain published 'Semi-Supervised Sequence Learning', showing that pre-training recurrent neural networks with unsupervised objectives, language modelling or sequence autoencoding, before supervised fine-tuning improved text classification accuracy and training stability, anticipating the pre-train-then-fine-tune paradigm later adopted widely in NLP.
Apple Releases Siri with iPhone 4S
On 4 October 2011, Apple announced Siri as an integrated feature of the iPhone 4S, making a conversational voice assistant, capable of natural-language queries, task execution, and third-party service calls, available to a mass consumer audience for the first time at that scale.
Long Short-Term Memory Introduced by Sepp Hochreiter and Jürgen Schmidhuber
In 1997, Sepp Hochreiter at Technische Universität München and Jürgen Schmidhuber at IDSIA published 'Long Short-Term Memory' in Neural Computation, introducing a recurrent neural network architecture with gated memory cells that could learn dependencies across long sequences without suffering from the vanishing gradient problem.