TD-Gammon Developed by Gerald Tesauro at IBM
In 1992, Gerald Tesauro at IBM Thomas J. Watson Research Center developed TD-Gammon, a backgammon program that trained itself through self-play using temporal-difference learning applied to a multilayer neural network, reaching a standard of play close to that of strong human experts.
Background
By the late 1980s, computer programs had conquered draughts and were closing in on chess, but backgammon sat in a different category. The game looks deceptively simple: roll dice, move pieces, try to bear them off the board before your opponent does. In practice, the number of positions the board can take is vast, and the dice introduce a layer of chance that makes it hard to evaluate any given position cleanly.
Programs at the time dealt with this by using hand-crafted evaluation functions. Programmers would study expert play, identify features that strong players cared about (prime formation, blot exposure, race equity), assign weights to those features, and build those judgements directly into the code. It worked up to a point. Programs could beat casual players and give intermediate players a reasonable game. But the features were only as good as the programmer’s understanding of backgammon theory, and that theory had limits. Anything the human experts had not already thought of could not get into the program.
The deeper problem was that no one knew how to learn a better evaluation function automatically. You could not simply collect labelled examples of positions with known correct values, because backgammon positions do not have tidy ground-truth values in the way, say, handwritten digits have correct labels.
What happened
Gerald Tesauro at IBM Thomas J. Watson Research Center took a different approach. Rather than encoding expert knowledge, he built a program that would teach itself by playing against its own previous versions, starting from almost nothing.
The system Tesauro built, TD-Gammon, combined two things. The first was a multilayer neural network, a layered arrangement of simple computational units loosely inspired by neurons, which took the current board position as input and produced an estimate of the probability of winning from that position. The second was temporal-difference learning, a method from reinforcement learning in which the program updates its estimates not by comparing against a known answer but by comparing what it predicted at one moment against what it predicted slightly later, gradually pushing those predictions into agreement with each other across a full game. The method had been formalised by Richard Sutton in the 1980s, but applying it to a real game of this complexity was new.
TD-Gammon started with random network weights and played against itself. It lost constantly at first. Over time, through hundreds of thousands of self-played games, the network’s position estimates became more accurate, and the program’s play improved. No human knowledge about backgammon strategy was written into the evaluation function; whatever the program learned, it found by playing.
The results were striking. Tesauro reported in his 1995 Communications of the ACM paper that the strongest version of TD-Gammon reached a level of play approaching world-class human standards. Independent experts who played against it agreed that it was playing at a very high level. Beyond raw performance, the program settled on some strategic choices, particularly around opening and mid-game cube handling, that differed from accepted human theory. Experienced players examined those choices and found them sound. The program had not just matched expert play; in certain respects it had found lines that the human community had not seriously considered before.
Why it mattered
TD-Gammon demonstrated that temporal-difference reinforcement learning, combined with a neural network function approximator and self-play, could produce expert-level performance in a complex board game without relying on hand-crafted evaluation functions. The result provided strong empirical evidence that reinforcement learning could scale to high-dimensional, real-world game domains, influencing subsequent work in game AI and, decades later, the design of systems such as AlphaGo. It also revealed strategic insights into backgammon not previously recognised by human players, illustrating that self-play could generate knowledge beyond existing human theory.
People
Organisations
IBM Thomas J Watson Research Center
Sources
- Temporal Difference Learning and TD-Gammon.Communications of the ACM.Primary source
- Practical Issues in Temporal Difference Learning.Springer.Primary source
- Practical Issues in Temporal Difference Learning.Machine Learning (Kluwer/Springer).Secondary
Cite this page
AI Achievements. (1992). TD-Gammon Developed by Gerald Tesauro at IBM. Retrieved 2026-08-22, from https://achievements.ai/milestone/td-gammon-program-gerald-tesauro
@misc{achievements_td_gammon_program_gerald_tesauro,
title = {TD-Gammon Developed by Gerald Tesauro at IBM},
author = {{AI Achievements}},
year = {1992},
url = {https://achievements.ai/milestone/td-gammon-program-gerald-tesauro}
}