AI milestones in 2022
13 documented milestones from 2022.
Self-Instruct: language models taught to follow instructions using synthetic self-generated data
Yizhong Wang and colleagues introduced Self-Instruct, a method for training language models to follow instructions using synthetic data the models generate themselves, closing much of the gap with InstructGPT-001 on evaluated tasks while using far less human annotation.
OpenAI launches ChatGPT as a free research preview
OpenAI released ChatGPT on 30 November 2022 as a free research preview built on GPT-3.5. It reached one million users within five days, and analysts at UBS estimated roughly 100 million monthly users by January 2023, a pace that had taken TikTok nine months and Instagram about two and a half years.
BLOOM: open-access 176B-parameter multilingual language model
The BigScience Workshop released BLOOM, a 176-billion-parameter language model trained across 46 natural and 13 programming languages, made freely available under the Responsible AI License at a scale that had only recently begun to become accessible, and had not previously been available with multilingual training data.
Flan-PaLM: instruction finetuning scaled across tasks, model sizes and families
Researchers showed that finetuning large language models on instruction-phrased datasets improves performance across benchmarks, with Flan-PaLM 540B trained on 1.8K tasks scoring 75.2% on five-shot MMLU and outperforming its base model by 9.4% on average.
ReAct: LLMs learn to interleave reasoning traces and actions
Shunyu Yao and colleagues introduced ReAct, a method letting large language models interleave reasoning traces with real actions such as querying Wikipedia, reducing hallucination and outperforming prior methods on question answering, fact verification and interactive decision-making tasks.
Stable Diffusion is released with downloadable weights
Stable Diffusion 1.4 was released publicly on 22 August 2022 with its weights freely downloadable, by Stability AI with CompVis at LMU Munich, Runway, EleutherAI and LAION. Because the diffusion runs in a compressed latent space, it generates 512 by 512 images in around 6.9 gigabytes of video memory, within reach of a consumer graphics card.
GitHub Copilot becomes generally available as a paid product
GitHub made Copilot generally available on 21 June 2022 at ten dollars a month, free to verified students and to maintainers of popular open source projects. It had run as a technical preview since June 2021 and suggests code inside the editor, drawing on Codex, a model trained on public code.
Flamingo: few-shot visual language model for interleaved images, video and text
Researchers introduced Flamingo, a family of Visual Language Models that could handle interleaved images, video and text, achieving state-of-the-art few-shot performance on many benchmarks without task-specific fine-tuning.
Anthropic paper applies RLHF to train helpful and harmless assistants
Researchers applied preference modelling and reinforcement learning from human feedback to fine-tune language models as assistants, finding that alignment training improved performance on almost all NLP evaluations tested, without measurable loss of specialised skills in the models studied.
SayCan: filtering language model suggestions through robot skill affordances
A team of researchers showed that combining a large language model with pretrained low-level robot skills lets a mobile manipulator complete long-horizon tasks from abstract natural language instructions, with the robot's physical capabilities constraining what the model is allowed to suggest.
InstructGPT: aligning language models with human feedback at scale
Researchers showed that fine-tuning GPT-3 with human feedback produced a 1.3B parameter model whose outputs labellers preferred over those of the 175B GPT-3, pointing toward a practical method for aligning language models more closely with expressed human preferences.
Chain-of-thought prompting improves step-by-step reasoning in large language models
Jason Wei and colleagues showed that prompting a large language model with worked examples of intermediate reasoning steps dramatically improved its performance on arithmetic, commonsense, and symbolic tasks, reaching state-of-the-art accuracy on a maths benchmark.
Multiresolution hash encoding cuts neural graphics training to seconds on a single GPU
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller introduced a multiresolution hash encoding that trains neural graphics primitives in seconds and renders at 1920×1080 in tens of milliseconds, achieving a combined speedup of several orders of magnitude when the encoding and optimised CUDA kernels are used together.