Reinforcement learning
14 milestones used this technique.
Process supervision outperforms outcome supervision for LLM reasoning
Researchers showed that giving a model step-by-step feedback on its reasoning, rather than judging only the final answer, raised the solve rate to 78% on a representative subset of the MATH benchmark, and released 800,000 human feedback labels to support further work.
Direct Preference Optimisation reduces need for RL in LM alignment
Rafael Rafailov and colleagues introduced Direct Preference Optimization (DPO), a method that aligns language models with human preferences using only a simple classification loss, bypassing the complex reinforcement learning pipeline that existing approaches required.
Anthropic paper applies RLHF to train helpful and harmless assistants
Researchers applied preference modelling and reinforcement learning from human feedback to fine-tune language models as assistants, finding that alignment training improved performance on almost all NLP evaluations tested, without measurable loss of specialised skills in the models studied.
SayCan: filtering language model suggestions through robot skill affordances
A team of researchers showed that combining a large language model with pretrained low-level robot skills lets a mobile manipulator complete long-horizon tasks from abstract natural language instructions, with the robot's physical capabilities constraining what the model is allowed to suggest.
InstructGPT: aligning language models with human feedback at scale
Researchers showed that fine-tuning GPT-3 with human feedback produced a 1.3B parameter model whose outputs labellers preferred over those of the 175B GPT-3, pointing toward a practical method for aligning language models more closely with expressed human preferences.
Reinforcement learning framework proposed to model T-cell adaptive immune response
In 2021, researchers published in Physical Review Research a theoretical framework proposing that T-cell receptor signalling during adaptive immunity can be formally described as a reinforcement learning process, connecting immunological learning to established machine-learning theory.
BWIBots: A Platform for Robots Operating in Human-Inhabited Buildings
In February 2017, researchers at the University of Texas at Austin's Building-Wide Intelligence project published a description of the BWIBots platform in the International Journal of Robotics Research, detailing mobile robots designed to operate autonomously in office buildings, accept natural-language commands, and learn cooperative tasks through human interaction.
OpenAI Released OpenAI Gym, a Toolkit for Reinforcement Learning Research
In April 2016, OpenAI publicly released OpenAI Gym, an open-source toolkit providing a standardised collection of environments for developing and benchmarking reinforcement learning algorithms, lowering the barrier to reproducible RL research.
DeepMind Publishes AlphaGo, a Deep Reinforcement Learning System That Defeated Professional Go Players
In January 2016, researchers at Google DeepMind published a paper in Nature describing AlphaGo, a system combining deep convolutional neural networks with Monte Carlo tree search and reinforcement learning that defeated the European Go champion Fan Hui 5–0, marking the first time a computer program had beaten a professional Go player at full-board Go.
DeepMind's DQN learns to play Atari games from raw pixels
DeepMind's Deep Q-Networks algorithm learned to play Atari 2600 games directly from raw pixels, matching or exceeding the score of a human tester on roughly half of the games tested, without any prior knowledge of the rules.
Schaft Inc Robot Wins DARPA Robotics Challenge Trials 2013
In December 2013, Schaft Inc, a Japanese robotics company acquired by Google in October 2013, won the DARPA Robotics Challenge Trials at Homestead Miami Speedway, Florida, scoring 27 out of 32 points across eight disaster-response tasks and finishing ahead of 15 other teams.
Zoe: Autonomous Astrobiology Field Robot for the Atacama Desert
In 2004, a team from Carnegie Mellon University's Field Robotics Center, NASA Ames Research Center, and the University of Tennessee deployed the Zoe rover autonomously across Chile's Atacama Desert, demonstrating robotic detection of subsurface biological life with relevance to astrobiology and future Mars exploration.
Fujitsu Laboratories Develops Dynamically Reconfigurable Neural Network for Humanoid Robot Motor Learning
In March 2003, Fujitsu Laboratories announced a dynamically reconfigurable neural network system enabling humanoid robots to learn motor coordination from experience, reducing learning time that previously required days or months and substantially cutting the volume of motion-control software needed.
TD-Gammon Developed by Gerald Tesauro at IBM
In 1992, Gerald Tesauro at IBM Thomas J. Watson Research Center developed TD-Gammon, a backgammon program that trained itself through self-play using temporal-difference learning applied to a multilayer neural network, reaching a standard of play close to that of strong human experts.