AI milestones in 2021
12 documented milestones from 2021.
S4: structured state spaces advance long-sequence modelling
Albert Gu, Karan Goel, and Christopher Ré introduced S4, a sequence model based on structured state spaces that achieved state-of-the-art results on the Long Range Arena benchmark and solved the Path-X task of length 16,000 steps that all prior work had failed.
Instruction tuning lets a 137B model match or beat GPT-3 zero-shot
Jason Wei and colleagues showed that finetuning a 137B language model on over 60 NLP tasks described via natural language instruction templates produced a model, FLAN, that beat zero-shot GPT-3 on 20 of 25 tasks and surpassed few-shot GPT-3 on several benchmarks.
RoseTTAFold reproduces protein structure prediction and releases the code
Baek and colleagues at the Institute for Protein Design published RoseTTAFold in Science on 15 July 2021, having rebuilt protein structure prediction from the ideas DeepMind had described at CASP14 eight months earlier without releasing. Its accuracy approached AlphaFold 2 without matching it, and it ran in about ten minutes on a gaming computer.
UMass Amherst researchers demonstrate protein-nanowire memristors for neuromorphic computing
In June 2021, researchers at the University of Massachusetts Amherst published findings in Nature Communications showing that protein nanowires harvested from the bacterium Geobacter sulfurreducens can function as memristors, enabling brain-inspired computation at ultralow power without a conventional battery.
BrainGate researchers decode imagined handwriting from neural signals to enable high-speed text communication
In May 2021, Francis R. Willett and colleagues at Stanford University and Howard Hughes Medical Institute published results showing that a BrainGate2 intracortical electrode array could decode imagined handwriting movements in a paralysed person at 90 characters per minute with 94.1% raw accuracy, substantially exceeding prior neural-interface typing rates.
Cornell University researchers demonstrate electrically actuated microscale origami robots with onboard CMOS control
On 17 March 2021, researchers at Cornell University published a demonstration of self-folding microscale robots, roughly 100–250 microns in size, driven by platinum-based shape-memory actuators and controlled by onboard complementary metal-oxide-semiconductor (CMOS) circuits, enabling untethered, electrically commanded origami-style locomotion at the micron scale.
OpenAI releases CLIP and DALL-E on the same day
OpenAI announced CLIP and DALL-E together on 5 January 2021. CLIP was trained on 400 million image and text pairs collected from the internet, and matches the accuracy of a ResNet-50 on ImageNet without using any of that benchmark's 1.28 million labelled examples. DALL-E generates images from a written description, using a version of GPT-3 adapted to produce pictures.
Reinforcement learning framework proposed to model T-cell adaptive immune response
In 2021, researchers published in Physical Review Research a theoretical framework proposing that T-cell receptor signalling during adaptive immunity can be formally described as a reinforcement learning process, connecting immunological learning to established machine-learning theory.
Google Announces TPU v4 Tensor Processing Unit
Google announced its fourth-generation Tensor Processing Unit (TPU v4) at Google I/O in 2021. The chip, designed specifically for large-scale machine-learning workloads, offered substantially higher performance than its predecessor and was made available to researchers via Google Cloud.
Cavatappi Artificial Muscles from Drawing, Twisting, and Coiling Polymer Tubes
In 2021, researchers led by Tzu-Ching Shyu at the University of Illinois at Urbana-Champaign published findings in Science Robotics describing cavatappi artificial muscles, twisted polymer actuators made from polyethylene-based tubes, capable of contracting up to 45% of their length, lifting approximately 300 times their weight per unit volume, and actuating within 23 milliseconds per 1% length change.
Pneumatic-logic soft robot without electronic components, University of California San Diego, 2021
In 2021, Michael T. Tolley and colleagues at the University of California San Diego published a soft walking robot controlled entirely by pneumatic logic circuits embedded in its body, with no electronic components, demonstrating autonomous gait and environmental responsiveness through fluidic computation alone.
Swinburne University of Technology Researchers Demonstrate Optical Neuromorphic Processor Using Micro-Comb Photonic Chip
In January 2021, a Swinburne University of Technology-led international team published results in Nature demonstrating an optical neuromorphic processor built on a photonic micro-comb chip, achieving a processing speed of 10 TOPS from a single integrated device and performing image classification tasks at high speed.