No. 312 of 500 No. 62 in Machine Learning

Algorithms for Decision Making

By Mykel J. Kochenderfer, Tim A. Wheeler and Kyle H. Wray

Print length
700 pages
Publisher
The MIT Press
Reading level
Advanced
ISBN-13
978-0262047012
Published
August 16, 2022
Written for
Graduate students, researchers and engineers building planning, reinforcement learning or autonomous decision systems

Synopsis

Mykel Kochenderfer, Tim Wheeler and Kyle Wray treat decision making under uncertainty as a single discipline and build it up in layers. Part one covers probabilistic reasoning: Bayesian networks, inference, parameter and structure learning, and utility-based choices made at a single moment. Part two moves to sequential problems, with exact and approximate dynamic programming, online planning, policy search, policy gradients and actor-critic methods. The remaining parts add model uncertainty, where an agent must learn by exploring; state uncertainty, handled through belief states and partially observable Markov decision processes; and multiagent settings, from game-theoretic reasoning to collaborative agents.

Every algorithm appears as compact Julia code beside the mathematics, and figures and exercises build intuition, so the book works both as a course text and as a reference for engineers. Examples include aircraft collision avoidance, a problem on which Kochenderfer's earlier work led to the ACAS X program. Appendices supply background in probability, computational complexity and neural representations. The book serves as the official lecture notes for Stanford's AA228/CS238 course, and a free PDF is available under a Creative Commons license. Readers gain a principled map connecting classical planning, modern reinforcement learning and the decision layer behind autonomous systems and AI agents.

Scorecard

Editorial score

How it scored

Editorial score: 78.0 / 1005 weighted criteria

  • Influence30%

    21.0/30
  • Authority25%

    22.5/25
  • Practical value20%

    16.0/20
  • Currency15%

    12.0/15
  • Readability10%

    6.5/10
  • Editorial score

    78.0/100

How the rank was computed

Rank No. 312 of 500 comes from a composite of 52.7: observable signals 17.6 of 55 points plus editorial review 35.1 of 45 points.

All 11 inputs
InputPoints
Observable signals17.6/55
Reach (Amazon ratings volume)5.5/16.5
Consensus across best-of lists5.5/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted5.6/8.3
Amazon sales rank1.0/5.5
Recency0.0/2.8
Editorial review35.1/45
Author authority10.1/11.3
Influence9.5/13.5
Practical value7.2/9.0
Readability2.9/4.5
Currency5.4/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A rigorous, freely available Stanford textbook that unifies planning, reinforcement learning and multiagent reasoning under uncertainty, with working Julia code.

Strengths

  • Free, continually corrected PDF edition
  • Unified notation across planning and learning
  • Concise Julia implementations of every algorithm

Keep in mind

Demands comfort with probability and mathematical notation, and Python-first readers must translate from Julia; deep reinforcement learning practice gets less attention than foundations.

Key takeaways

  1. Most decision problems under uncertainty can be framed as Markov decision processes or their extensions, giving a common language for planning and learning.
  2. Reinforcement learning is presented as a response to model uncertainty, where an agent must balance exploring its environment against exploiting what it knows.
  3. When the true state is hidden, agents should plan over belief states, a framework that underlies robust robotics and autonomous navigation.
  4. Multiagent problems require game-theoretic reasoning, since each agent's best action depends on the choices and information of others.

Notable facts

  • Stanford's AA228/CS238 course uses the book as its textbook and official lecture notes. Source (opens in a new tab)
  • The full book is available as a free PDF under a Creative Commons CC BY-NC-ND license, while MIT Press sells the print edition. Source (opens in a new tab)
  • Lead author Kochenderfer's earlier work at MIT Lincoln Laboratory on aircraft collision avoidance led to the establishment of the ACAS X program. Source (opens in a new tab)

About the author

Mykel J. Kochenderfer, Tim A. Wheeler and Kyle H. Wray

Mykel J. Kochenderfer is an associate professor of aeronautics and astronautics at Stanford, director of the Stanford Intelligent Systems Laboratory and a senior fellow at Stanford HAI. At MIT Lincoln Laboratory, his work on aircraft collision avoidance led to the ACAS X program. He holds an Edinburgh PhD and co-wrote the book with Tim A. Wheeler and Kyle H. Wray.

Questions

Is Algorithms for Decision Making free?
Yes. The authors and MIT Press offer the complete book as a free PDF at algorithmsbook.com under a Creative Commons noncommercial, no-derivatives license, and they update it as errors are reported. MIT Press sells a 700-page print edition for readers who prefer a physical copy. Supporting material, including slides, is maintained in a public GitHub repository.
Who should read Algorithms for Decision Making?
It suits graduate students and advanced undergraduates in computer science, engineering or operations research, along with engineers building autonomous systems, robotics or AI agents. Readers need working knowledge of probability, linear algebra and programming. It is less suited to beginners seeking an intuitive introduction to AI, and Python users should expect to read Julia code, which is concise and easy to translate.
How does Algorithms for Decision Making relate to reinforcement learning?
Reinforcement learning is one of its central topics, but the book places it inside a broader framework. It first covers planning when the environment model is known, then treats reinforcement learning as the response to model uncertainty, covering exploration, model-based and model-free methods and imitation learning, before extending to hidden states and multiple agents.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.