Human Compatible
Artificial Intelligence and the Problem of Control
Synopsis
Stuart Russell, coauthor of the field's standard textbook, argues that AI research rests on a flawed premise he calls the standard model: give a machine a fixed objective and let it optimize. As systems grow more capable, he warns, a mis-specified objective becomes dangerous, because a machine pursuing the wrong goal has every reason to resist correction and to treat being switched off as an obstacle. Russell explains what intelligence means in humans and machines, which breakthroughs still separate current AI from superhuman systems, and how AI is already being misused for surveillance, persuasion and autonomous weapons.
His alternative rests on three principles: a machine's only purpose is to realize human preferences, it starts out uncertain about what those preferences are, and it learns them by observing human behavior. A machine built this way has a reason to ask permission and accept shutdown. Russell rebuts prominent skeptics of AI risk, works through the complications of serving many people with conflicting and imperfect preferences, and adds appendices on search, logic, probability and learning. Readers gain a precise vocabulary for the control problem and a research agenda that still shapes alignment work.
Scorecard
Editorial score
How it scored
Editorial score: 80.3 / 1005 weighted criteria
- 25.5/30
Influence30%
- 23.8/25
Authority25%
- 13.0/20
Practical value20%
- 10.5/15
Currency15%
- 7.5/10
Readability10%
- 80.3/100
Editorial score
How the rank was computed
Rank No. 27 of 500 comes from a composite of 73.5: observable signals 37.3 of 55 points plus editorial review 36.2 of 45 points.
All 11 inputs
| Input | Points |
|---|---|
| Observable signals | 37.3/55 |
| Reach (Amazon ratings volume) | 10.8/16.5 |
| Consensus across best-of lists | 13.8/13.8 |
| Awards and bestseller evidence | 5.0/8.3 |
| Reader rating, sample-adjusted | 6.1/8.3 |
| Amazon sales rank | 1.6/5.5 |
| Recency | 0.0/2.8 |
| Editorial review | 36.2/45 |
| Author authority | 10.7/11.3 |
| Influence | 11.5/13.5 |
| Practical value | 5.9/9.0 |
| Readability | 3.4/4.5 |
| Currency | 4.7/6.8 |
Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.
Verdict
The most rigorous general-audience case that AI must be built to defer to uncertain human preferences, written by a field-defining researcher.
Strengths
- Written by a foremost AI researcher
- Concrete technical proposal, not only warnings
- Engages skeptics' arguments point by point
Keep in mind
Published before the ChatGPT era, it says little about how today's chatbots are trained, and skeptics of superintelligence, including some reviewers, reject its premises.
Key takeaways
- Building machines that optimize fixed objectives is risky, because any mis-specified goal pursued by a capable system can produce harmful side effects.
- Beneficial machines should be uncertain about human preferences, which gives them an incentive to defer, ask permission and accept shutdown.
- Inverse reinforcement learning, which infers goals from observed behavior, offers one route for machines to learn what people actually want.
- Near-term misuse, including mass surveillance, behavioral manipulation and lethal autonomous weapons, demands attention alongside long-term control risks.
Notable facts
- Longlisted for the 2019 Financial Times and McKinsey Business Book of the Year Award. Source (opens in a new tab)
- The Guardian's Ian Sample called it the most important book on AI that year. Source (opens in a new tab)
- Russell's textbook Artificial Intelligence: A Modern Approach, coauthored with Peter Norvig, has been adopted by more than 1,500 schools. Source (opens in a new tab)
About the author
Questions
What is Human Compatible about?
Is Human Compatible still relevant in the age of ChatGPT?
Do I need a technical background to read Human Compatible?
Related books
Sources
As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.






