Reinforcement Learning
An Introduction
Synopsis
Richard Sutton and Andrew Barto present reinforcement learning as the study of agents that learn by trial and error to maximize reward while interacting with an uncertain environment. First published in 1998 and substantially expanded in a 2018 second edition, the book builds from simple cases to general methods. Part I covers problems small enough for exact tabular solutions: multi-armed bandits, Markov decision processes, dynamic programming, Monte Carlo methods, temporal-difference learning and planning with learned models. Part II extends those ideas to function approximation, where values and policies are represented by linear methods or neural networks, and treats off-policy learning, eligibility traces and policy-gradient methods, including actor-critic algorithms.
Part III connects the algorithms to psychology and neuroscience, including the link between temporal-difference errors and dopamine signals, and surveys applications such as TD-Gammon, Atari game playing, and DeepMind's AlphaGo and AlphaGo Zero. The authors favor intuition and pseudocode, setting heavier mathematics apart in shaded boxes, and include exercises throughout, which suits classroom use. The book does not cover recent large-scale deep RL engineering or reinforcement learning from human feedback as used to tune large language models. Readers come away with the conceptual grammar of value functions, bootstrapping and exploration that underlies those later techniques.
Scorecard
Editorial score
How it scored
Editorial score: 86.8 / 1005 weighted criteria
- 28.5/30
Influence30%
- 25.0/25
Authority25%
- 15.0/20
Practical value20%
- 11.3/15
Currency15%
- 7.0/10
Readability10%
- 86.8/100
Editorial score
How the rank was computed
Rank No. 70 of 500 comes from a composite of 65.9: observable signals 26.7 of 55 points plus editorial review 39.2 of 45 points.
All 11 inputs
| Input | Points |
|---|---|
| Observable signals | 26.7/55 |
| Reach (Amazon ratings volume) | 10.3/16.5 |
| Consensus across best-of lists | 8.3/13.8 |
| Awards and bestseller evidence | 0.0/8.3 |
| Reader rating, sample-adjusted | 6.7/8.3 |
| Amazon sales rank | 1.4/5.5 |
| Recency | 0.0/2.8 |
| Editorial review | 39.2/45 |
| Author authority | 11.3/11.3 |
| Influence | 12.8/13.5 |
| Practical value | 6.8/9.0 |
| Readability | 3.2/4.5 |
| Currency | 5.1/6.8 |
Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.
Verdict
The standard reinforcement learning textbook, rigorous yet readable, written by the Turing Award winners who established the field's foundations.
Strengths
- Authoritative coverage from the field's founders
- Clear intuition backed by pseudocode and exercises
- Links algorithms to psychology and neuroscience
Keep in mind
Requires comfort with probability and calculus, and readers wanting modern deep RL implementations or RLHF for language models will need supplementary sources.
Key takeaways
- Reinforcement learning frames intelligence as an agent learning from rewards through interaction, distinct from supervised learning on labeled examples.
- Temporal-difference learning updates predictions from other predictions, combining Monte Carlo sampling with the bootstrapping of dynamic programming to learn efficiently online.
- Balancing exploration of new actions against exploitation of known rewards is a central, recurring challenge across every family of algorithms.
- Function approximation and policy-gradient methods scale tabular ideas to large problems, forming the bridge to deep reinforcement learning systems such as AlphaGo.
Notable facts
- Sutton and Barto received the 2024 ACM A.M. Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning. Source (opens in a new tab)
- The book is described as the field's standard reference and has been cited more than 75,000 times. Source (opens in a new tab)
About the author
Questions
Is Reinforcement Learning: An Introduction good for beginners?
Which edition of Reinforcement Learning should I read?
Is Reinforcement Learning: An Introduction still relevant in 2026?
Related books
- An Introduction to Statistical Learning Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani and Jonathan Taylor
- The Master Algorithm Pedro Domingos
- The Elements of Statistical Learning Trevor Hastie, Robert Tibshirani and Jerome Friedman
- Why Machines Learn Anil Ananthaswamy
- Pattern Recognition and Machine Learning Christopher M. Bishop
- Data Science for Business Foster Provost and Tom Fawcett
Sources
As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.






