GPT-1
OpenAI's 2018 language model showing that unsupervised pre-training on large text corpora improves downstream NLP tasks.
1 milestone
OpenAI Releases GPT-1: Improving Language Understanding by Generative Pre-Training
In June 2018, Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever at OpenAI published 'Improving Language Understanding by Generative Pre-Training', introducing GPT-1, a 117-million-parameter Transformer pretrained on BooksCorpus via unsupervised language modelling and fine-tuned on downstream tasks, outperforming task-specific models on several NLP benchmarks.