Backpropagation Described by Arthur E. Bryson Jr. and Yu-Chi Ho
In 1969, Arthur E. Bryson Jr. and Yu-Chi Ho of Harvard University described a gradient-based optimisation procedure for multi-stage dynamic systems in their textbook Applied Optimal Control, presenting what is now recognised as an early statement of the backpropagation principle in a supervised-learning context.
Background
By the late 1960s, researchers in control engineering had a well-developed interest in optimisation: given a system that evolves through several steps, how do you find the inputs that bring it closest to some desired outcome? The tools available were largely borrowed from classical mechanics and variational calculus, a branch of mathematics concerned with finding paths or functions that minimise some quantity. These methods worked, but applying them to systems with many stages and many variables was slow and, in practice, often intractable.
Neural networks existed as a concept. Frank Rosenblatt’s perceptron, built in the late 1950s, had shown that a simple network of artificial neurones could learn to classify inputs. But the perceptron could only learn one layer at a time. Training a network with multiple layers meant adjusting weights across those layers simultaneously, and nobody had a clean, general method for deciding how much blame each weight deserved when the network made an error. Without that, deep networks were mostly a theoretical curiosity.
The missing piece was a way to work backwards. Given an error at the end of a multi-stage process, you needed a systematic method for tracing that error back through each stage and calculating how much each earlier decision had contributed to it. Gradient descent, the general idea of repeatedly nudging parameters in whichever direction reduces the error, was understood. The difficulty was computing those gradients efficiently when the stages were chained together.
What happened
Arthur Earl Bryson Jr. and Yu-Chi Ho were both at Harvard University in 1969 when Blaisdell Publishing released their textbook Applied Optimal Control: Optimization, Estimation, and Control. The book was aimed at engineers working on control systems, not at the nascent field of machine learning. Its subject was how to optimise a process that unfolds across multiple stages, the kind of problem that arises in aerospace guidance or economic planning.
Within that framework, Bryson and Ho described a gradient-based procedure for minimising a cost function, a numerical measure of how far a system’s output is from the desired result, by propagating gradient information backward through the stages of the system. The mathematics applied the chain rule of calculus in sequence, from the final stage back to the first. That is the same operation that would later be called backpropagation when applied to neural networks.
The textbook did not use the word “backpropagation”, and Bryson and Ho were not writing about neural networks. Their result sat in the control-theory literature and was largely unknown to the researchers who, in the 1980s, were trying to train multi-layer networks. Bryson and Ho were also not alone: Henry J. Kelley had described a related approach in 1960, Stuart Dreyfus had published a closely related derivation in 1962, and Seppo Linnainmaa would independently describe the reverse accumulation of gradients in 1970. The 1986 paper by David Rumelhart, Geoffrey Hinton, and Ronald Williams in Nature is the work that made the method widely known to the machine learning community, and that paper does not cite Bryson and Ho. The attribution of priority to the 1969 textbook is a conclusion drawn largely by historians of the field, not a connection Bryson and Ho made themselves.
Why it mattered
Bryson and Ho's formulation showed that gradients could be propagated backward through a multi-stage system to minimise a cost function, providing a mathematically rigorous foundation for what would later become the central training algorithm of artificial neural networks. Although the result was not widely applied to neural networks until Rumelhart, Hinton, and Williams popularised the method in 1986, the 1969 text is consistently cited by historians of machine learning as an early, independent derivation. Recognising this priority clarifies the intellectual lineage of deep learning and corrects the common misconception that backpropagation originated entirely in the 1980s.
People
Arthur E Bryson Jr, Yu-Chi Ho
Organisations
Sources
- Applied Optimal Control: Optimization, Estimation, and Control, review notice and bibliographic record.Blaisdell Publishing / Ginn and Company.Primary source
- Learning representations by back-propagating errors.Nature.Secondary
- Who Invented Backpropagation?.IDSIA, Dalle Molle Institute for Artificial Intelligence.Secondary
Cite this page
AI Achievements. (1969). Backpropagation Described by Arthur E. Bryson Jr. and Yu-Chi Ho. Retrieved 2026-08-22, from https://achievements.ai/milestone/backpropagation-by-arthur-yu-chi-ho
@misc{achievements_backpropagation_by_arthur_yu_chi_ho,
title = {Backpropagation Described by Arthur E. Bryson Jr. and Yu-Chi Ho},
author = {{AI Achievements}},
year = {1969},
url = {https://achievements.ai/milestone/backpropagation-by-arthur-yu-chi-ho}
}