No. 70 of 500 No. 4 in Machine Learning

Reinforcement Learning

An Introduction

By Richard S. Sutton and Andrew G. Barto

Print length
552 pages
Publisher
Bradford Books
Reading level
Advanced
ISBN-13
978-0262039246
Published
November 13, 2018
Written for
Graduate students, researchers and engineers who want a rigorous foundation in reinforcement learning algorithms.

Synopsis

Richard Sutton and Andrew Barto present reinforcement learning as the study of agents that learn by trial and error to maximize reward while interacting with an uncertain environment. First published in 1998 and substantially expanded in a 2018 second edition, the book builds from simple cases to general methods. Part I covers problems small enough for exact tabular solutions: multi-armed bandits, Markov decision processes, dynamic programming, Monte Carlo methods, temporal-difference learning and planning with learned models. Part II extends those ideas to function approximation, where values and policies are represented by linear methods or neural networks, and treats off-policy learning, eligibility traces and policy-gradient methods, including actor-critic algorithms.

Part III connects the algorithms to psychology and neuroscience, including the link between temporal-difference errors and dopamine signals, and surveys applications such as TD-Gammon, Atari game playing, and DeepMind's AlphaGo and AlphaGo Zero. The authors favor intuition and pseudocode, setting heavier mathematics apart in shaded boxes, and include exercises throughout, which suits classroom use. The book does not cover recent large-scale deep RL engineering or reinforcement learning from human feedback as used to tune large language models. Readers come away with the conceptual grammar of value functions, bootstrapping and exploration that underlies those later techniques.

Scorecard

Editorial score

How it scored

Editorial score: 86.8 / 1005 weighted criteria

  • Influence30%

    28.5/30
  • Authority25%

    25.0/25
  • Practical value20%

    15.0/20
  • Currency15%

    11.3/15
  • Readability10%

    7.0/10
  • Editorial score

    86.8/100

How the rank was computed

Rank No. 70 of 500 comes from a composite of 65.9: observable signals 26.7 of 55 points plus editorial review 39.2 of 45 points.

All 11 inputs
InputPoints
Observable signals26.7/55
Reach (Amazon ratings volume)10.3/16.5
Consensus across best-of lists8.3/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted6.7/8.3
Amazon sales rank1.4/5.5
Recency0.0/2.8
Editorial review39.2/45
Author authority11.3/11.3
Influence12.8/13.5
Practical value6.8/9.0
Readability3.2/4.5
Currency5.1/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

The standard reinforcement learning textbook, rigorous yet readable, written by the Turing Award winners who established the field's foundations.

Strengths

  • Authoritative coverage from the field's founders
  • Clear intuition backed by pseudocode and exercises
  • Links algorithms to psychology and neuroscience

Keep in mind

Requires comfort with probability and calculus, and readers wanting modern deep RL implementations or RLHF for language models will need supplementary sources.

Key takeaways

  1. Reinforcement learning frames intelligence as an agent learning from rewards through interaction, distinct from supervised learning on labeled examples.
  2. Temporal-difference learning updates predictions from other predictions, combining Monte Carlo sampling with the bootstrapping of dynamic programming to learn efficiently online.
  3. Balancing exploration of new actions against exploitation of known rewards is a central, recurring challenge across every family of algorithms.
  4. Function approximation and policy-gradient methods scale tabular ideas to large problems, forming the bridge to deep reinforcement learning systems such as AlphaGo.

Notable facts

  • Sutton and Barto received the 2024 ACM A.M. Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning. Source (opens in a new tab)
  • The book is described as the field's standard reference and has been cited more than 75,000 times. Source (opens in a new tab)

About the author

Richard S. Sutton and Andrew G. Barto

Richard S. Sutton is a professor of computing science at the University of Alberta, affiliated with the Alberta Machine Intelligence Institute, and a former distinguished research scientist at DeepMind. Andrew G. Barto, his doctoral adviser, is professor emeritus at the University of Massachusetts Amherst. The two shared the 2024 ACM A.M. Turing Award for developing the foundations of reinforcement learning.

Questions

Is Reinforcement Learning: An Introduction good for beginners?
It is introductory in the sense that it assumes no prior reinforcement learning knowledge, but it is a university-level text. Readers should be comfortable with probability, basic linear algebra and mathematical notation. Newcomers without that background often start with an online course or a hands-on tutorial and then turn to Sutton and Barto for depth and precise definitions.
Which edition of Reinforcement Learning should I read?
Read the second edition, published by MIT Press in 2018. It substantially expands the 1998 original with new algorithms such as Expected Sarsa and Double Learning, a full part on function approximation and policy-gradient methods, chapters on psychology and neuroscience, and case studies including AlphaGo and AlphaGo Zero. The first edition is now mainly of historical interest.
Is Reinforcement Learning: An Introduction still relevant in 2026?
Yes. Reinforcement learning underpins game-playing systems, robotics research and the post-training of large language models, and those methods rest on the value functions, temporal-difference learning and policy gradients this book explains. It does not cover reinforcement learning from human feedback or recent large-scale engineering practice, so practitioners should pair it with current papers and documentation.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.