Gerald Tesauro
Gerald J. "Gerry" Tesauro | |
|---|---|
| Alma mater | University of Maryland, College Park; Princeton University |
| Occupation(s) | Computer scientist, researcher |
| Known for | TD-Gammon, autonomic computing, IBM Watson strategic algorithms |
| Awards | Hertz Foundation Fellowship (1980), AAAI Fellow (2013), ACM Fellow (2018) |
Gerald J. "Gerry" Tesauro is a distinguished American computer scientist and researcher at IBM, renowned for creating TD-Gammon, a backgammon software that attained mastery at the world championship level by employing self-play alongside temporal difference learning. This achievement represented a pioneering advancement in reinforcement learning and neural networks. Subsequently, he made significant contributions to autonomic computing, multi-agent systems within the realm of e-commerce, and the strategic algorithms utilized by IBM Watson.
Education
Tesauro obtained his Bachelor of Science degree in physics from the University of Maryland, College Park. He then advanced to graduate studies in plasma physics at Princeton University, supported by a prestigious Hertz Foundation Fellowship beginning in 1980. He earned his Ph.D. in theoretical physics in 1986 under the mentorship of Nobel Prize winner Philip W. Anderson.
Backgammon
Following his doctoral studies, Tesauro undertook postdoctoral work at the Center for Complex Systems Research at the University of Illinois Urbana-Champaign. There, he applied neural network techniques to gaming, co-authoring a 1987 NeurIPS publication with Terrence Sejnowski on a neural network designed to learn backgammon.
By the late 1980s, Tesauro joined the IBM's Thomas J. Watson Research Center as a research scientist and later advanced to Principal Research Staff Member specializing in AI Science.
During this period, he developed Neurogammon, a backgammon program trained through supervised learning on expert-level games. Neurogammon clinched victory at the first Computer Olympiad backgammon tournament in 1989, demonstrating the effectiveness of neural networks in game artificial intelligence.
From 1990 until 1998, Tesauro engineered TD-Gammon, which leveraged reinforcement learning via temporal-difference (TD) methods. This program refined its play by self-competition, employing neural networks to assess board positions. TD-Gammon reached performance comparable to world champions and was capable of competing with elite human players. It is widely regarded as a seminal success in both machine learning and reinforcement learning fields and served as an inspiration for subsequent systems such as AlphaZero.
In this timeframe, Tesauro also contributed to IBM’s computer chess initiatives by applying machine learning techniques to optimize evaluation function parameters, especially those related to king safety, although leadership of the Deep Blue project lay elsewhere. Since 2010, he has been involved with the computer Go project Fuego, further advancing AI strategies for complex games.
E-commerce
Toward the late 1990s, Tesauro shifted focus toward multi-agent systems tailored for applications in e-commerce. He developed autonomous "pricebots"—software agents capable of learning optimal pricing and bidding approaches within electronic marketplaces. Employing Q-learning, these agents adapted dynamic pricing strategies such as cooperation or competitive undercutting. This work represented one of the earliest applications of multi-agent reinforcement learning to economic modeling and automated trading. Additionally, Tesauro investigated neural network methods for detecting computer virus threats.
Autonomic computing
Beginning in the early 2000s, Tesauro was instrumental in advancing IBM's autonomic computing program, which sought to develop IT systems capable of self-management. He utilized reinforcement learning techniques to automate tasks such as resource distribution, system performance optimization, and energy control within data centers and distributed networks. His research involved orchestrating multiple reinforcement learning agents to enhance server resource utilization—including CPU, memory, and power consumption—aiming for improved operational efficiency and reduced energy use.
Between 2004 and 2007, Tesauro secured a significant number of U.S. patents centered on autonomic computing and AI-driven systems management. These patents address methods for reward-based learning of system policies, dynamic allocation of resources guided by utility functions, and the transfer of autonomic models across different computing platforms.
IBM Watson
In approximately 2009, Tesauro became part of the IBM Research team under David Ferrucci that created IBM Watson, the advanced question-answering system famous for its 2011 victory over human champions Ken Jennings and Brad Rutter on Jeopardy!.
Tesauro contributed specifically to Watson’s strategic gameplay modules, developing algorithms responsible for buzzer timing, clue selection, and wagering strategies—particularly regarding Daily Doubles and Final Jeopardy!. Alongside his colleagues, he designed a Game State Evaluator employing simulation-based optimization alongside principles from Bayesian inference, game theory, dynamic programming, and reinforcement learning to refine Watson’s decision-making. These innovations were critical in enabling Watson to manage risk effectively and execute near-optimal betting strategies.
During this timeframe, Tesauro also co-authored a paper on Monte Carlo Simulation Balancing with David Silver, presented at ICML 2009. Following his work on Watson, he continued his research at IBM focusing on areas such as deep reinforcement learning, hierarchical reinforcement learning, multi-agent frameworks, and lifelong learning.
Honors and awards
- Recipient of the Hertz Foundation Fellowship (Class of 1980)
- Elected Fellow of the Association for the Advancement of Artificial Intelligence (AAAI) in 2013 for his impactful contributions to neural computation, game-playing AI (including Backgammon, Chess, and Jeopardy!), autonomic computing, and economic agent design
- Named Fellow of the Association for Computing Machinery (ACM) in 2018 in recognition of his pioneering work in reinforcement learning, neural networks, and intelligent autonomous agents
External links
- Gerald Tesauro profile at Chess Programming Wiki
- Gerald Tesauro publication list on DBLP