Language models
21 milestones used this technique.
Mixtral 8x7B: sparse mixture-of-experts model released under Apache 2.0
Mixtral 8x7B, a sparse mixture-of-experts language model, was published in January 2024. It outperforms or matches Llama 2 70B and GPT-3.5 on most evaluated benchmarks while using only 13B active parameters per token, and both base and instruction-tuned versions were released under the Apache 2.0 licence.
Process supervision outperforms outcome supervision for LLM reasoning
Researchers showed that giving a model step-by-step feedback on its reasoning, rather than judging only the final answer, raised the solve rate to 78% on a representative subset of the MATH benchmark, and released 800,000 human feedback labels to support further work.
Direct Preference Optimisation reduces need for RL in LM alignment
Rafael Rafailov and colleagues introduced Direct Preference Optimization (DPO), a method that aligns language models with human preferences using only a simple classification loss, bypassing the complex reinforcement learning pipeline that existing approaches required.
QLoRA enables finetuning of 65B-parameter models on a single 48GB GPU
Tim Dettmers and colleagues showed that a 65-billion-parameter language model could be finetuned on a single 48GB GPU without measurable quality loss on the benchmarks tested, by combining 4-bit quantisation with low-rank adapter training.
LLaMA matches leading models on public data with far fewer parameters
LLaMA, a collection of foundation language models from 7B to 65B parameters trained exclusively on publicly available data, was submitted to arXiv on 27 February 2023. The 13B model outperformed GPT-3 at 175B parameters on most benchmarks, and the weights were released to the research community.
Self-Instruct: language models taught to follow instructions using synthetic self-generated data
Yizhong Wang and colleagues introduced Self-Instruct, a method for training language models to follow instructions using synthetic data the models generate themselves, closing much of the gap with InstructGPT-001 on evaluated tasks while using far less human annotation.
BLOOM: open-access 176B-parameter multilingual language model
The BigScience Workshop released BLOOM, a 176-billion-parameter language model trained across 46 natural and 13 programming languages, made freely available under the Responsible AI License at a scale that had only recently begun to become accessible, and had not previously been available with multilingual training data.
Flan-PaLM: instruction finetuning scaled across tasks, model sizes and families
Researchers showed that finetuning large language models on instruction-phrased datasets improves performance across benchmarks, with Flan-PaLM 540B trained on 1.8K tasks scoring 75.2% on five-shot MMLU and outperforming its base model by 9.4% on average.
ReAct: LLMs learn to interleave reasoning traces and actions
Shunyu Yao and colleagues introduced ReAct, a method letting large language models interleave reasoning traces with real actions such as querying Wikipedia, reducing hallucination and outperforming prior methods on question answering, fact verification and interactive decision-making tasks.
Anthropic paper applies RLHF to train helpful and harmless assistants
Researchers applied preference modelling and reinforcement learning from human feedback to fine-tune language models as assistants, finding that alignment training improved performance on almost all NLP evaluations tested, without measurable loss of specialised skills in the models studied.
SayCan: filtering language model suggestions through robot skill affordances
A team of researchers showed that combining a large language model with pretrained low-level robot skills lets a mobile manipulator complete long-horizon tasks from abstract natural language instructions, with the robot's physical capabilities constraining what the model is allowed to suggest.
InstructGPT: aligning language models with human feedback at scale
Researchers showed that fine-tuning GPT-3 with human feedback produced a 1.3B parameter model whose outputs labellers preferred over those of the 175B GPT-3, pointing toward a practical method for aligning language models more closely with expressed human preferences.
Chain-of-thought prompting improves step-by-step reasoning in large language models
Jason Wei and colleagues showed that prompting a large language model with worked examples of intermediate reasoning steps dramatically improved its performance on arithmetic, commonsense, and symbolic tasks, reaching state-of-the-art accuracy on a maths benchmark.
Instruction tuning lets a 137B model match or beat GPT-3 zero-shot
Jason Wei and colleagues showed that finetuning a 137B language model on over 60 NLP tasks described via natural language instruction templates produced a model, FLAN, that beat zero-shot GPT-3 on 20 of 25 tasks and surpassed few-shot GPT-3 on several benchmarks.
OpenAI released GPT-3 via private beta API
In May–June 2020, OpenAI published the GPT-3 language model in a paper by Tom B. Brown and colleagues, and began distributing private beta API access. GPT-3's 175 billion parameters made it substantially larger than any publicly described language model at the time, enabling strong few-shot performance across diverse language tasks.
Microsoft Research released Turing Natural Language Generation (T-NLG), a 17-billion-parameter language model
In February 2020, Microsoft Research announced Turing Natural Language Generation (T-NLG), a 17-billion-parameter autoregressive language model trained using the Megatron-LM framework. At the time of release it was the largest publicly disclosed language model and achieved state-of-the-art results on question-answering and summarisation benchmarks.
OpenAI Released the Full 1.5-Billion-Parameter GPT-2 Model
In November 2019, OpenAI released the full 1.5-billion-parameter version of GPT-2, completing a staged release the organisation had begun in February 2019 with a smaller variant, citing concerns about potential misuse of a model capable of generating coherent long-form text.
OpenAI Releases GPT-1: Improving Language Understanding by Generative Pre-Training
In June 2018, Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever at OpenAI published 'Improving Language Understanding by Generative Pre-Training', introducing GPT-1, a 117-million-parameter Transformer pretrained on BooksCorpus via unsupervised language modelling and fine-tuned on downstream tasks, outperforming task-specific models on several NLP benchmarks.
CIMON Launched to the International Space Station
In June 2018, DLR, Airbus, and IBM launched CIMON (Crew Interactive Mobile Companion), a spherical, voice-controlled AI assistant, to the International Space Station aboard SpaceX CRS-15. It was designed to support ESA astronaut Alexander Gerst with procedural tasks and reduce cognitive workload.
Andrew M. Dai and Quoc V. Le Introduced Semi-Supervised Sequence Learning
In November 2015, Andrew M. Dai and Quoc V. Le at Google Brain published 'Semi-Supervised Sequence Learning', showing that pre-training recurrent neural networks with unsupervised objectives, language modelling or sequence autoencoding, before supervised fine-tuning improved text classification accuracy and training stability, anticipating the pre-train-then-fine-tune paradigm later adopted widely in NLP.
Amazon Introduces Alexa and the Echo Smart Speaker
In November 2014, Amazon introduced Alexa, a cloud-based voice assistant, alongside the Amazon Echo smart speaker. Alexa accepted natural-language voice commands for tasks including music playback, information retrieval, smart-home control, and calendar management, bringing always-on voice interaction into consumer homes at scale.