Language models
7 milestones used this technique.
OpenAI released GPT-3 via private beta API
In May–June 2020, OpenAI published the GPT-3 language model in a paper by Tom B. Brown and colleagues, and began distributing private beta API access. GPT-3's 175 billion parameters made it substantially larger than any publicly described language model at the time, enabling strong few-shot performance across diverse language tasks.
Microsoft Research released Turing Natural Language Generation (T-NLG), a 17-billion-parameter language model
In February 2020, Microsoft Research announced Turing Natural Language Generation (T-NLG), a 17-billion-parameter autoregressive language model trained using the Megatron-LM framework. At the time of release it was the largest publicly disclosed language model and achieved state-of-the-art results on question-answering and summarisation benchmarks.
OpenAI Released the Full 1.5-Billion-Parameter GPT-2 Model
In November 2019, OpenAI released the full 1.5-billion-parameter version of GPT-2, completing a staged release the organisation had begun in February 2019 with a smaller variant, citing concerns about potential misuse of a model capable of generating coherent long-form text.
OpenAI Releases GPT-1: Improving Language Understanding by Generative Pre-Training
In June 2018, Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever at OpenAI published 'Improving Language Understanding by Generative Pre-Training', introducing GPT-1, a 117-million-parameter Transformer pretrained on BooksCorpus via unsupervised language modelling and fine-tuned on downstream tasks, outperforming task-specific models on several NLP benchmarks.
CIMON Launched to the International Space Station
In June 2018, DLR, Airbus, and IBM launched CIMON (Crew Interactive Mobile Companion), a spherical, voice-controlled AI assistant, to the International Space Station aboard SpaceX CRS-15. It was designed to support ESA astronaut Alexander Gerst with procedural tasks and reduce cognitive workload.
Andrew M. Dai and Quoc V. Le Introduced Semi-Supervised Sequence Learning
In November 2015, Andrew M. Dai and Quoc V. Le at Google Brain published 'Semi-Supervised Sequence Learning', showing that pre-training recurrent neural networks with unsupervised objectives, language modelling or sequence autoencoding, before supervised fine-tuning improved text classification accuracy and training stability, anticipating the pre-train-then-fine-tune paradigm later adopted widely in NLP.
Amazon Introduces Alexa and the Echo Smart Speaker
In November 2014, Amazon introduced Alexa, a cloud-based voice assistant, alongside the Amazon Echo smart speaker. Alexa accepted natural-language voice commands for tasks including music playback, information retrieval, smart-home control, and calendar management, bringing always-on voice interaction into consumer homes at scale.