AI research
Milestones where artificial intelligence itself is the subject: architectures, algorithms, training methods and the theory behind them.
107 milestones
OpenAI releases o1, a model trained to reason before answering
OpenAI released o1-preview and o1-mini on 12 September 2024, models trained by reinforcement learning to produce an extended chain of thought before answering. On the 2024 AIME examination GPT-4o averaged 12 per cent of problems; o1 averaged 74 per cent with a single attempt and 93 per cent when re-ranking a thousand samples.
GPT-4o handles speech, vision and text in one network
OpenAI released GPT-4o on 13 May 2024, trained end to end across text, vision and audio so that every input and output passes through one network. It responds to speech in as little as 232 milliseconds and 320 on average, against roughly 2.8 seconds for the three separate models it replaced.
Mixtral 8x7B: sparse mixture-of-experts model released under Apache 2.0
Mixtral 8x7B, a sparse mixture-of-experts language model, was published in January 2024. It outperforms or matches Llama 2 70B and GPT-3.5 on most evaluated benchmarks while using only 13B active parameters per token, and both base and instruction-tuned versions were released under the Apache 2.0 licence.
3D Gaussian Splatting achieves real-time novel-view synthesis at 1080p
Bernhard Kerbl and colleagues applied 3D Gaussian splatting to novel-view synthesis with end-to-end optimisation and adaptive density control, achieving high-quality results at 1080p resolution and real-time frame rates, without the slow neural rendering that previous approaches required.
FlashAttention-2 roughly doubles attention speed on A100 GPUs
Tri Dao's FlashAttention-2 improved GPU work partitioning to deliver roughly 2x the speed of FlashAttention, reaching 50–73% of the theoretical maximum FLOPs/s in the configurations benchmarked in the paper, and up to 225 TFLOPs/s per A100 in the specific GPT-style training configurations reported.
Process supervision outperforms outcome supervision for LLM reasoning
Researchers showed that giving a model step-by-step feedback on its reasoning, rather than judging only the final answer, raised the solve rate to 78% on a representative subset of the MATH benchmark, and released 800,000 human feedback labels to support further work.
Direct Preference Optimisation reduces need for RL in LM alignment
Rafael Rafailov and colleagues introduced Direct Preference Optimization (DPO), a method that aligns language models with human preferences using only a simple classification loss, bypassing the complex reinforcement learning pipeline that existing approaches required.
QLoRA enables finetuning of 65B-parameter models on a single 48GB GPU
Tim Dettmers and colleagues showed that a 65-billion-parameter language model could be finetuned on a single 48GB GPU without measurable quality loss on the benchmarks tested, by combining 4-bit quantisation with low-rank adapter training.
OpenAI releases GPT-4, the first of its models to accept images
OpenAI released GPT-4 on 14 March 2023, a model accepting both image and text input and producing text. Its technical report gives human-level performance on a range of professional and academic examinations, including a simulated bar exam in roughly the top ten per cent of test takers.
LLaMA matches leading models on public data with far fewer parameters
LLaMA, a collection of foundation language models from 7B to 65B parameters trained exclusively on publicly available data, was submitted to arXiv on 27 February 2023. The 13B model outperformed GPT-3 at 175B parameters on most benchmarks, and the weights were released to the research community.
ControlNet adds structured spatial conditioning to pretrained text-to-image diffusion models
Lvmin Zhang, Anyi Rao and Maneesh Agrawala introduced ControlNet, a neural network architecture that adds spatial conditioning controls such as edges, depth, segmentation and human pose to large pretrained text-to-image diffusion models without degrading their existing capabilities.
Self-Instruct: language models taught to follow instructions using synthetic self-generated data
Yizhong Wang and colleagues introduced Self-Instruct, a method for training language models to follow instructions using synthetic data the models generate themselves, closing much of the gap with InstructGPT-001 on evaluated tasks while using far less human annotation.
BLOOM: open-access 176B-parameter multilingual language model
The BigScience Workshop released BLOOM, a 176-billion-parameter language model trained across 46 natural and 13 programming languages, made freely available under the Responsible AI License at a scale that had only recently begun to become accessible, and had not previously been available with multilingual training data.
Flan-PaLM: instruction finetuning scaled across tasks, model sizes and families
Researchers showed that finetuning large language models on instruction-phrased datasets improves performance across benchmarks, with Flan-PaLM 540B trained on 1.8K tasks scoring 75.2% on five-shot MMLU and outperforming its base model by 9.4% on average.
ReAct: LLMs learn to interleave reasoning traces and actions
Shunyu Yao and colleagues introduced ReAct, a method letting large language models interleave reasoning traces with real actions such as querying Wikipedia, reducing hallucination and outperforming prior methods on question answering, fact verification and interactive decision-making tasks.
Stable Diffusion is released with downloadable weights
Stable Diffusion 1.4 was released publicly on 22 August 2022 with its weights freely downloadable, by Stability AI with CompVis at LMU Munich, Runway, EleutherAI and LAION. Because the diffusion runs in a compressed latent space, it generates 512 by 512 images in around 6.9 gigabytes of video memory, within reach of a consumer graphics card.
Flamingo: few-shot visual language model for interleaved images, video and text
Researchers introduced Flamingo, a family of Visual Language Models that could handle interleaved images, video and text, achieving state-of-the-art few-shot performance on many benchmarks without task-specific fine-tuning.
Anthropic paper applies RLHF to train helpful and harmless assistants
Researchers applied preference modelling and reinforcement learning from human feedback to fine-tune language models as assistants, finding that alignment training improved performance on almost all NLP evaluations tested, without measurable loss of specialised skills in the models studied.
InstructGPT: aligning language models with human feedback at scale
Researchers showed that fine-tuning GPT-3 with human feedback produced a 1.3B parameter model whose outputs labellers preferred over those of the 175B GPT-3, pointing toward a practical method for aligning language models more closely with expressed human preferences.
Chain-of-thought prompting improves step-by-step reasoning in large language models
Jason Wei and colleagues showed that prompting a large language model with worked examples of intermediate reasoning steps dramatically improved its performance on arithmetic, commonsense, and symbolic tasks, reaching state-of-the-art accuracy on a maths benchmark.
Multiresolution hash encoding cuts neural graphics training to seconds on a single GPU
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller introduced a multiresolution hash encoding that trains neural graphics primitives in seconds and renders at 1920×1080 in tens of milliseconds, achieving a combined speedup of several orders of magnitude when the encoding and optimised CUDA kernels are used together.
S4: structured state spaces advance long-sequence modelling
Albert Gu, Karan Goel, and Christopher Ré introduced S4, a sequence model based on structured state spaces that achieved state-of-the-art results on the Long Range Arena benchmark and solved the Path-X task of length 16,000 steps that all prior work had failed.
Instruction tuning lets a 137B model match or beat GPT-3 zero-shot
Jason Wei and colleagues showed that finetuning a 137B language model on over 60 NLP tasks described via natural language instruction templates produced a model, FLAN, that beat zero-shot GPT-3 on 20 of 25 tasks and surpassed few-shot GPT-3 on several benchmarks.
UMass Amherst researchers demonstrate protein-nanowire memristors for neuromorphic computing
In June 2021, researchers at the University of Massachusetts Amherst published findings in Nature Communications showing that protein nanowires harvested from the bacterium Geobacter sulfurreducens can function as memristors, enabling brain-inspired computation at ultralow power without a conventional battery.
OpenAI releases CLIP and DALL-E on the same day
OpenAI announced CLIP and DALL-E together on 5 January 2021. CLIP was trained on 400 million image and text pairs collected from the internet, and matches the accuracy of a ResNet-50 on ImageNet without using any of that benchmark's 1.28 million labelled examples. DALL-E generates images from a written description, using a version of GPT-3 adapted to produce pictures.
Google Announces TPU v4 Tensor Processing Unit
Google announced its fourth-generation Tensor Processing Unit (TPU v4) at Google I/O in 2021. The chip, designed specifically for large-scale machine-learning workloads, offered substantially higher performance than its predecessor and was made available to researchers via Google Cloud.
Swinburne University of Technology Researchers Demonstrate Optical Neuromorphic Processor Using Micro-Comb Photonic Chip
In January 2021, a Swinburne University of Technology-led international team published results in Nature demonstrating an optical neuromorphic processor built on a photonic micro-comb chip, achieving a processing speed of 10 TOPS from a single integrated device and performing image classification tasks at high speed.
OpenAI released GPT-3 via private beta API
In May–June 2020, OpenAI published the GPT-3 language model in a paper by Tom B. Brown and colleagues, and began distributing private beta API access. GPT-3's 175 billion parameters made it substantially larger than any publicly described language model at the time, enabling strong few-shot performance across diverse language tasks.
Once-for-All: Train One Network and Specialize It for Efficient Deployment
Han Cai, Chuang Gan, Tianhao Chen, and Song Han at MIT published Once-for-All at ICLR 2020, presenting a method to train a single neural network once and then derive specialised sub-networks for diverse hardware platforms without retraining, reducing the computational cost of neural architecture search by orders of magnitude.
Microsoft Research released Turing Natural Language Generation (T-NLG), a 17-billion-parameter language model
In February 2020, Microsoft Research announced Turing Natural Language Generation (T-NLG), a 17-billion-parameter autoregressive language model trained using the Megatron-LM framework. At the time of release it was the largest publicly disclosed language model and achieved state-of-the-art results on question-answering and summarisation benchmarks.
OpenAI Released the Full 1.5-Billion-Parameter GPT-2 Model
In November 2019, OpenAI released the full 1.5-billion-parameter version of GPT-2, completing a staged release the organisation had begun in February 2019 with a smaller variant, citing concerns about potential misuse of a model capable of generating coherent long-form text.
Facebook AI Research Releases Detectron2
In October 2019, Facebook AI Research released Detectron2, an open-source object detection and segmentation framework built on PyTorch, supporting algorithms including Mask R-CNN, DensePose, and panoptic feature pyramid networks, replacing the earlier Caffe2-based Detectron.
Analogue resistive memory circuit solves linear algebra problems in one step
In March 2019, Daniele Ielmini and colleagues at Politecnico di Milano published results in PNAS demonstrating a crosspoint resistive-memory circuit that solves linear systems, matrix eigenvector problems, and differential equations by physical analogue relaxation, substantially reducing the energy and latency costs of conventional iterative digital solvers.
BERT introduces masked bidirectional pre-training for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova introduced BERT, a language model that pre-trains on both left and right context simultaneously, achieving new best results on eleven natural language processing tasks.
OpenAI Releases GPT-1: Improving Language Understanding by Generative Pre-Training
In June 2018, Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever at OpenAI published 'Improving Language Understanding by Generative Pre-Training', introducing GPT-1, a 117-million-parameter Transformer pretrained on BooksCorpus via unsupervised language modelling and fine-tuned on downstream tasks, outperforming task-specific models on several NLP benchmarks.
Facebook AI Research Published StarSpace: Embed All The Things!
In September 2017, Ledell Wu and colleagues at Facebook AI Research published StarSpace (arXiv:1709.03856), a general-purpose neural embedding model capable of learning entity representations across tasks including text classification, ranking, and collaborative filtering, without task-specific architecture changes.
Movidius (Intel) Launches Neural Compute Stick
In July 2017, Movidius, an Intel subsidiary, released the Movidius Neural Compute Stick, a USB-form-factor device housing the Myriad 2 Vision Processing Unit, enabling developers to run inference from trained deep neural networks on low-power edge hardware without a remote server.
Vaswani et al. introduce the Transformer, a sequence model built on self-attention alone
Ashish Vaswani and colleagues proposed the Transformer, a sequence model whose encoding and decoding relied on multi-head self-attention rather than recurrence or convolutions. It set new translation benchmarks while training faster than existing models.
FAISS: Facebook AI Research Library for Efficient Similarity Search
In 2017, Jeff Johnson, Matthijs Douze, and Hervé Jégou at Facebook AI Research published FAISS (Facebook AI Similarity Search), a library enabling efficient nearest-neighbour search across datasets of billions of vectors, with GPU acceleration substantially reducing search time compared to prior methods.
WaveNet: A Generative Model for Raw Audio, by DeepMind
In September 2016, researchers at Google DeepMind published WaveNet, a deep generative model that synthesises raw audio waveforms sample-by-sample using dilated causal convolutions. In evaluations on English and Mandarin speech, WaveNet reduced the gap between human speech and machine synthesis by more than 50 per cent compared with the best previous text-to-speech systems.
OpenAI Released OpenAI Gym, a Toolkit for Reinforcement Learning Research
In April 2016, OpenAI publicly released OpenAI Gym, an open-source toolkit providing a standardised collection of environments for developing and benchmarking reinforcement learning algorithms, lowering the barrier to reproducible RL research.
Deep residual networks make training at 100+ layers practical with identity shortcuts
Kaiming He and colleagues introduced residual learning, letting networks train at depths of up to 152 layers. Their ResNet won first place on five tracks at the ILSVRC and COCO 2015 competitions, including ImageNet classification, detection, localisation, and COCO detection and segmentation.
OpenAI Founded
In December 2015, a group of technology investors and researchers (including Greg Brockman, Ilya Sutskever, Wojciech Zaremba, John Schulman, Elon Musk, and Sam Altman) announced the founding of OpenAI, a non-profit artificial intelligence research laboratory in San Francisco, with approximately one billion US dollars in pledged funding.
Andrew M. Dai and Quoc V. Le Introduced Semi-Supervised Sequence Learning
In November 2015, Andrew M. Dai and Quoc V. Le at Google Brain published 'Semi-Supervised Sequence Learning', showing that pre-training recurrent neural networks with unsupervised objectives, language modelling or sequence autoencoding, before supervised fine-tuning improved text classification accuracy and training stability, anticipating the pre-train-then-fine-tune paradigm later adopted widely in NLP.
DeepMind's DQN learns to play Atari games from raw pixels
DeepMind's Deep Q-Networks algorithm learned to play Atari 2600 games directly from raw pixels, matching or exceeding the score of a human tester on roughly half of the games tested, without any prior knowledge of the rules.
Kingma and Ba present the Adam optimiser
Diederik P. Kingma and Jimmy Ba presented Adam, an optimisation algorithm combining and extending existing adaptive gradient methods, adapting learning rates using bias-corrected estimates of the first and second moments of gradients, making it efficient and broadly practical, particularly where heavy hyperparameter tuning is not feasible.
Neural Turing Machine Introduced by Alex Graves, Greg Wayne, and Ivo Danihelka
In October 2014, Alex Graves, Greg Wayne, and Ivo Danihelka at Google DeepMind published 'Neural Turing Machines', a preprint proposing a neural network architecture augmented with an external memory matrix and differentiable read/write operations, enabling the system to learn algorithms such as sorting and copying from examples alone.
Bahdanau, Cho and Bengio introduce soft attention for neural machine translation
Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio proposed an attention mechanism letting neural translation models search source sentences dynamically, rather than compressing everything into a single fixed-length vector, achieving performance comparable to phrase-based systems on English-to-French translation.
IBM TrueNorth: a neuromorphic integrated circuit with one million programmable neurons
In August 2014, researchers at IBM Research published a description of TrueNorth, a neuromorphic chip containing one million programmable spiking neurons and 256 million synapses on a 4096-core CMOS integrated circuit, built under the DARPA SyNAPSE programme.
Goodfellow and colleagues propose generative adversarial networks
Ian Goodfellow and seven co-authors proposed training two neural networks against each other: one generating samples, one judging them. The setup, requiring only backpropagation and no Markov chains, could recover the training data distribution under idealised theoretical assumptions.
Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta at Carnegie Mellon University present NEIL (Never-Ending Image Learner) at ICCV 2013
In December 2013, Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta at Carnegie Mellon University presented NEIL (Never-Ending Image Learner) at ICCV 2013, a continuously running system that autonomously mined semantic relationships between visual concepts from unlabelled web images without human supervision.
Google, NASA and USRA Launch Quantum Artificial Intelligence Lab
In May 2013, Google, NASA Ames Research Center and the Universities Space Research Association jointly established the Quantum Artificial Intelligence Lab at NASA's Ames facility, housing a D-Wave Two quantum processor to investigate whether quantum annealing could accelerate machine-learning tasks.
AlexNet and Deep Convolutional Neural Networks in Large-Scale Image Classification
In September 2012, Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton at the University of Toronto submitted a paper describing AlexNet, a deep convolutional neural network that achieved a top-5 error rate of 15.3% on the ImageNet Large Scale Visual Recognition Challenge, outperforming the next-best entry by more than 10 percentage points.
Dropout introduced as a regularisation method for neural networks
Srivastava, Hinton, Krizhevsky, Sutskever and Salakhutdinov introduced dropout, a method that randomly omits units during training to stop neural networks from overfitting. It set new records on several specific speech and object recognition benchmarks at the time of publication.
Google Brain Unsupervised Neural Network Learns to Detect Cats from YouTube Frames
In June 2012, Quoc V. Le and colleagues at Google Brain published research showing that a 1,000-machine, 16,000-core neural network trained without labels on 10 million YouTube thumbnail images spontaneously developed a neuron selectively responsive to human and cat faces, demonstrating large-scale unsupervised feature learning from unlabelled video data.
Google Brain Founded by Andrew Ng and Jeff Dean
In 2011, Andrew Ng and Jeff Dean co-founded Google Brain, an internal research group at Google dedicated to large-scale deep learning. The project demonstrated that deep neural networks trained on substantial compute could learn useful representations without labelled data, reshaping how the industry approached machine learning research.
DeepMind Technologies Founded
Demis Hassabis, Shane Legg, and Mustafa Suleyman co-founded DeepMind Technologies in London in 2010, establishing an independent research laboratory with the stated goal of developing general-purpose artificial intelligence grounded in neuroscience.
ImageNet: a large-scale hierarchical image database for object recognition
Researchers led by Li Fei-Fei introduced ImageNet, a dataset organised around more than 100,000 concept categories and aimed at providing roughly 1,000 human-annotated images per category, on which the ILSVRC challenge was later built.
OpenCog Artificial General Intelligence Framework Introduced
In 2008, Ben Goertzel and colleagues at the Singularity Institute for Artificial Intelligence publicly introduced OpenCog, an open-source software framework designed to support research into artificial general intelligence by integrating multiple cognitive subsystems within a shared knowledge store called the AtomSpace.
Numenta Founded by Jeff Hawkins and Donna Dubinsky
In 2005, Jeff Hawkins and Donna Dubinsky co-founded Numenta, a research company dedicated to developing machine intelligence systems modelled on the structural and algorithmic principles of the mammalian neocortex, building on Hawkins's theoretical framework published in his 2004 book On Intelligence.
Rapid Object Detection Using a Boosted Cascade of Simple Features (Viola–Jones Face Detection)
Paul Viola and Michael Jones, then at Compaq CRL and Mitsubishi Electric Research Laboratories respectively, published a cascaded boosting framework for real-time face detection, first presented at CVPR in December 2001 and consolidated in the International Journal of Computer Vision in 2004. The method ran at frame rates suitable for live video on consumer hardware.
MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) Founded
In July 2003, MIT merged its Laboratory for Computer Science (LCS) and its Artificial Intelligence Laboratory (AI Lab) to form the Computer Science and Artificial Intelligence Laboratory (CSAIL), creating a single research organisation that brought together computing systems and AI research under one institutional structure.
Bag of Words Applied to Computer Vision (Visual Vocabulary / Bag of Visual Words)
Josef Sivic and Andrew Zisserman at the University of Oxford applied the Bag of Words text-retrieval model to visual features in their 2003 ICCV paper 'Video Google', representing image regions as a vocabulary of visual words to enable efficient object retrieval from video.
A Neural Probabilistic Language Model by Yoshua Bengio and Colleagues
In 2003, Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin at the Université de Montréal published 'A Neural Probabilistic Language Model' in JMLR, demonstrating that a feed-forward neural network trained on word sequences could learn distributed word representations and outperform n-gram models on perplexity benchmarks.
Structured Light for Robust Correspondence in Active Stereo Vision
In 2002, Li Zhang, Brian Curless, and Steven M. Seitz at the University of Washington presented a method using structured light patterns projected onto scenes to establish robust stereo correspondences, enabling reliable 3D reconstruction under conditions where passive stereo fails.
Torch Machine Learning Library
In 2002, Ronan Collobert, Samy Bengio, and Johnny Mariéthoz at IDIAP Research Institute published a paper introducing Torch, a modular C++ and Lua-scriptable machine learning library that unified a range of algorithms, including support vector machines and neural networks, under a common object-oriented framework.
NVIDIA GeForce 256: Introduction of the Graphics Processing Unit (GPU)
In 1999, NVIDIA released the GeForce 256, which the company marketed as the first graphics processing unit (GPU), a single-chip processor capable of performing transform, lighting, clipping, and rendering operations that had previously required the host CPU, enabling sustained high-throughput parallel computation.
Long Short-Term Memory Introduced by Sepp Hochreiter and Jürgen Schmidhuber
In 1997, Sepp Hochreiter at Technische Universität München and Jürgen Schmidhuber at IDSIA published 'Long Short-Term Memory' in Neural Computation, introducing a recurrent neural network architecture with gated memory cells that could learn dependencies across long sequences without suffering from the vanishing gradient problem.
ALICE Chatbot Created by Richard S. Wallace
In 1995, Richard S. Wallace, an independent AI researcher, created ALICE (Artificial Linguistic Internet Computer Entity), a natural-language chatbot that used a pattern-matching markup language called AIML to generate contextually plausible conversational responses, later influencing a generation of open-source chatbot development.
Python 0.9.0 Released by Guido van Rossum
In February 1991, Guido van Rossum, then at Centrum Wiskunde & Informatica in Amsterdam, publicly released Python 0.9.0 by posting it to the alt.sources newsgroup. The language offered an accessible, readable syntax and became foundational infrastructure for scientific computing and, later, machine learning research and tooling.
Elephants Don't Play Chess by Rodney Brooks
In 1990, Rodney Brooks of MIT's Artificial Intelligence Laboratory published 'Elephants Don't Play Chess' in Robotics and Autonomous Systems, arguing that classical symbolic AI was fundamentally misconceived and that intelligence emerges from direct physical interaction with the environment rather than from abstract symbol manipulation.
Rodney Brooks Publishes 'Elephants Don't Play Chess', Articulating Nouvelle AI
In 1990, Rodney Brooks of MIT published 'Elephants Don't Play Chess' in Robotics and Autonomous Systems, arguing that intelligent behaviour could emerge from direct sensorimotor coupling with the environment without internal symbolic representations, formalising the nouvelle AI research programme.
Jabberwacky Chatbot Developed by Rollo Carpenter
In 1988, British programmer Rollo Carpenter began developing Jabberwacky, a chatbot that simulated conversation by storing and retrieving lines from prior user exchanges rather than using fixed scripted responses, with the aim of exploring machine-based natural language interaction.
IBM TJ Watson Research Center Publishes Statistical Approach to Machine Translation
In August 1988, researchers at IBM Thomas J. Watson Research Center, including Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, Robert L. Mercer, and Paul S. Roossin, presented a statistical framework for machine translation at COLING 1988, replacing rule-based linguistics with probabilistic models trained on bilingual text corpora.
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference Published by Judea Pearl
In 1988, Judea Pearl of UCLA published Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference with Morgan Kaufmann, providing a systematic framework for representing and computing with uncertain knowledge using Bayesian networks and belief propagation algorithms.
NETtalk Neural Network Developed by Terrence J. Sejnowski and Charles Rosenberg
Terrence J. Sejnowski of the Salk Institute and Charles Rosenberg of Princeton University developed NETtalk, a feedforward neural network trained to convert English text to speech, publishing the principal account in Complex Systems in 1987. The network learned pronunciation from examples alone, demonstrating that a multi-layer perceptron could acquire a complex linguistic skill without hand-coded rules.
Second AI Winter
From approximately 1987 to 1993, AI research entered a second sustained contraction as commercial expert-system vendors collapsed, the Lisp machine market failed, and DARPA substantially reduced funding for AI programmes following unmet expectations from the preceding boom.
W. Daniel Hillis Proposes the Connection Machine Architecture
In 1985, W. Daniel Hillis of MIT and Thinking Machines Corporation completed his doctoral dissertation introducing the Connection Machine, a massively parallel architecture connecting 65,536 single-bit processors to accelerate symbolic and artificial-intelligence computation, realised as the CM-1 system.
HAM-ANS Natural Language Dialog System (University of Hamburg, 1983)
In 1983, Wolfgang Hoeppner, Katharina Morik, and Heinz Marburger at the University of Hamburg presented HAM-ANS, a natural language dialog system enabling users to query databases in ordinary German, demonstrated at the GWAI-83 workshop and published in Springer's Lecture Notes in Computer Science series.
SOAR Cognitive Architecture: Doctoral Dissertations by John E. Laird and Paul S. Rosenbloom, Supervised by Allen Newell
In 1983, John E. Laird and Paul S. Rosenbloom completed doctoral dissertations at Carnegie Mellon University under Allen Newell, introducing SOAR, a cognitive architecture designed to support a broad range of intelligent tasks through a unified problem-space model and a chunking-based learning mechanism.
Primal Sketch Theory of Early Visual Representation Described by David Marr
David Marr, working at MIT's Artificial Intelligence Laboratory, formalised the primal sketch as the first stage of his three-level theory of visual processing, published posthumously in 'Vision' (1982). The model proposed that the visual system constructs a symbolic, viewer-centred description of intensity changes and local geometry before any object recognition takes place.
Fifth Generation Computer Systems Project, Japan
In October 1981, Japan's Ministry of International Trade and Industry launched the Fifth Generation Computer Systems project, a ten-year state-funded programme delegated to the newly created Institute for New Generation Computer Technology (ICOT), aiming to build computers capable of logical inference, natural-language processing, and machine vision, with a budget of approximately 54 billion yen.
Blackboard Model Description by Lee Erman, Richard Hayes-Roth, Victor Lesser and D. Raj Reddy
In May 1980, Lee Erman, Richard Hayes-Roth, Victor Lesser and D. Raj Reddy published 'The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty' in Artificial Intelligence, vol. 14, providing the canonical description of the blackboard model as a structured framework for cooperative problem-solving among independent knowledge sources.
John Searle Publishes the Chinese Room Argument
In 1980, John Searle at the University of California, Berkeley published 'Minds, Brains, and Programs' in Behavioral and Brain Sciences, presenting the Chinese Room thought experiment to argue that executing a computer program is insufficient to produce understanding or intentionality, directly challenging claims of strong artificial intelligence.
Commercial Lisp Machine Market: Symbolics and LMI
From 1980 onwards, Symbolics Inc. and Lisp Machines Inc. (LMI), both founded as spin-offs from the MIT Artificial Intelligence Laboratory, commercialised dedicated hardware workstations designed to run Lisp natively, offering large address spaces, garbage collection in hardware, and early graphical interfaces tailored to AI development.
Meta-Level Knowledge in Expert Systems: Davis and Lenat's Formalisation at Stanford
In 1977, Randall Davis and Douglas Lenat at Stanford University published research formalising meta-level knowledge, a system's explicit representations of its own knowledge and reasoning strategies, and demonstrated its application in the MYCIN and AM expert systems to improve inference control and self-directed learning.
ABSTRIPS Developed by Earl Sacerdoti at SRI International
In 1974, Earl Sacerdoti at SRI International published ABSTRIPS, an extension of the STRIPS planning system that organised problem-solving into a hierarchy of abstraction spaces, allowing a planner to resolve high-level constraints before committing to fine-grained detail.
Human Associative Memory (HAM) Model Published by John R. Anderson and Gordon H. Bower
In 1973, John R. Anderson and Gordon H. Bower, both at Stanford University, published 'Human Associative Memory', introducing the HAM model, a propositional network architecture representing semantic memory as binary trees, providing a computationally explicit theory of human memory that influenced subsequent cognitive architectures.
Prolog Logic Programming Language Created by Alain Colmerauer and Philippe Roussel
In 1972, Alain Colmerauer and Philippe Roussel at the University of Aix-Marseille created Prolog (Programmation en Logique), a declarative programming language grounded in first-order predicate logic, enabling computers to reason over symbolic knowledge without requiring procedural step-by-step instructions.
Augmented Transition Networks Introduced by William A. Woods
In 1970, William A. Woods of Bolt Beranek and Newman published 'Transition Network Grammars for Natural Language Analysis' in Communications of the ACM, introducing Augmented Transition Networks (ATNs) as a formalism for parsing natural language by extending finite-state transition networks with recursion and registers, enabling more expressive grammatical coverage.
Stanford Heuristic Programming Project founded by Edward Feigenbaum
In 1970, Edward Feigenbaum founded the Heuristic Programming Project (HPP) at Stanford University, establishing a dedicated research group to investigate the construction of knowledge-based expert systems and their application to scientific and medical domains.
SHRDLU Natural Language Understanding Program Developed by Terry Winograd at MIT
In 1970, Terry Winograd at the Massachusetts Institute of Technology completed SHRDLU, a natural language understanding program that allowed a user to converse in English about a simulated world of coloured blocks, demonstrating that a computer could parse and respond to complex grammatical instructions within a constrained domain.
Backpropagation Described by Arthur E. Bryson Jr. and Yu-Chi Ho
In 1969, Arthur E. Bryson Jr. and Yu-Chi Ho of Harvard University described a gradient-based optimisation procedure for multi-stage dynamic systems in their textbook Applied Optimal Control, presenting what is now recognised as an early statement of the backpropagation principle in a supervised-learning context.
Perceptrons: An Introduction to Computational Geometry
In 1969, Marvin Minsky and Seymour Papert of MIT published Perceptrons: An Introduction to Computational Geometry, a formal mathematical analysis of single-layer perceptrons that demonstrated key limitations, notably the inability to compute non-linearly separable functions such as XOR, and contributed to a reduction in funding and research activity in connectionist approaches to AI.
Stanford Computer Forum Founded as Industry–Academia Bridge
In May 1968, Stanford professors Ed McCluskey, Arthur Samuel, and William Miller founded the Stanford Computer Forum, an industrial affiliates programme linking the university's electrical engineering and computer science research, including AI work, with corporate partners across Silicon Valley.
A* Search Algorithm Published by Hart, Nilsson, and Raphael at Stanford Research Institute
In 1968, Peter E. Hart, Nils J. Nilsson, and Bertram Raphael at the Stanford Research Institute published 'A Formal Basis for the Heuristic Determination of Minimum Cost Paths', introducing the A* search algorithm, which finds shortest paths in graphs efficiently by combining actual path cost with a heuristic estimate of remaining cost.
Analogy: A Program That Solves Geometric Analogy Problems by Thomas C. Evans
Thomas C. Evans, working at MIT, developed ANALOGY, a program capable of solving geometric analogy problems of the type found in standard IQ tests. First presented in dissertation form in 1963 and formally published in 1968, it demonstrated that a computer could perform a structured form of relational reasoning.
ELIZA Developed by Joseph Weizenbaum at MIT
In January 1966, Joseph Weizenbaum of MIT published a paper in Communications of the ACM describing ELIZA, a computer program that simulated conversation by applying pattern-matching rules to user input, and documented the unexpected tendency of human users to attribute understanding and empathy to the system.
Alchemy and Artificial Intelligence, RAND Corporation Memorandum by Hubert Dreyfus
In December 1965, Hubert Dreyfus, a philosopher at the Massachusetts Institute of Technology consulting for the RAND Corporation, published RAND Memorandum P-3244, 'Alchemy and Artificial Intelligence', arguing that the cognitive assumptions underlying contemporary AI research were philosophically untenable and that the field faced fundamental, not merely technical, limits.
I. J. Good Defines the Ultraintelligent Machine and Articulates the Intelligence Explosion
In 1965, statistician I. J. Good, then at Trinity College Oxford, published 'Speculations Concerning the First Ultraintelligent Machine' in Advances in Computers, vol. 6, defining an ultraintelligent machine as one that surpasses all human intellectual activity and arguing that building such a machine would trigger an 'intelligence explosion.'
Herbert A. Simon predicts machines will be capable of any work a human can do, within twenty years
In 1965, Herbert A. Simon of Carnegie Mellon University published 'The Shape of Automation for Men and Management', in which he predicted that machines would, within twenty years, be capable of performing any cognitive task a human could perform, a claim that became one of the most cited and scrutinised forecasts in the history of artificial intelligence.
Machine Perception of Three-Dimensional Solids, Lawrence Gilman Roberts (MIT Lincoln Laboratory)
In 1963, Lawrence Gilman Roberts, working at MIT Lincoln Laboratory, completed his doctoral thesis demonstrating that a computer could interpret a 2D photograph of polyhedral objects, reconstruct their 3D structure, and re-render them from arbitrary viewpoints with hidden lines removed, establishing foundational methods for machine interpretation of three-dimensional scenes.
IBM Shoebox Speech-Recognition Device Demonstrated
In 1962, IBM publicly demonstrated the Shoebox, a hardware device capable of recognising sixteen spoken English words, the digits zero through nine plus six arithmetic commands, and using that input to drive a calculating machine in real time, showing that limited-vocabulary speech recognition was mechanically feasible.
Man-Computer Symbiosis, paper by J. C. R. Licklider
In March 1960, J. C. R. Licklider, then at Bolt Beranek and Newman, published 'Man-Computer Symbiosis' in IRE Transactions on Human Factors in Electronics, articulating a vision in which humans and computers would collaborate interactively in real time to solve problems neither could address alone.
General Problem Solver introduced by Newell, Shaw and Simon
In 1959, Allen Newell and Herbert A. Simon at the RAND Corporation and Carnegie Institute of Technology, with J. C. Shaw, presented the General Problem Solver, a computer program that separated problem-solving strategy from domain knowledge using means–ends analysis.
John McCarthy Coins the Term 'Artificial Intelligence' in the Dartmouth Conference Proposal
In 1955, John McCarthy of Dartmouth College, together with Marvin Minsky, Nathaniel Rochester, and Claude Shannon, submitted a proposal to the Rockefeller Foundation for a summer research workshop, introducing the term 'artificial intelligence' and framing machine intelligence as a formal field of scientific inquiry.
Programming a Computer for Playing Chess, Claude Shannon
In March 1950, Claude Shannon, then at Bell Telephone Laboratories, published 'Programming a Computer for Playing Chess' in Philosophical Magazine, outlining two strategic approaches, exhaustive search (Type A) and selective heuristic search (Type B), that framed computer chess research for decades.