No. 27 of 500 No. 8 in AI Safety & the Future

Human Compatible

Artificial Intelligence and the Problem of Control

By Stuart Russell

Print length
352 pages
Publisher
Penguin Books
Reading level
Intermediate
ISBN-13
978-0525558637
Published
November 17, 2020
Written for
Readers wanting a rigorous yet accessible foundation in AI alignment and the control problem.

Synopsis

Stuart Russell, coauthor of the field's standard textbook, argues that AI research rests on a flawed premise he calls the standard model: give a machine a fixed objective and let it optimize. As systems grow more capable, he warns, a mis-specified objective becomes dangerous, because a machine pursuing the wrong goal has every reason to resist correction and to treat being switched off as an obstacle. Russell explains what intelligence means in humans and machines, which breakthroughs still separate current AI from superhuman systems, and how AI is already being misused for surveillance, persuasion and autonomous weapons.

His alternative rests on three principles: a machine's only purpose is to realize human preferences, it starts out uncertain about what those preferences are, and it learns them by observing human behavior. A machine built this way has a reason to ask permission and accept shutdown. Russell rebuts prominent skeptics of AI risk, works through the complications of serving many people with conflicting and imperfect preferences, and adds appendices on search, logic, probability and learning. Readers gain a precise vocabulary for the control problem and a research agenda that still shapes alignment work.

Scorecard

Editorial score

How it scored

Editorial score: 80.3 / 1005 weighted criteria

  • Influence30%

    25.5/30
  • Authority25%

    23.8/25
  • Practical value20%

    13.0/20
  • Currency15%

    10.5/15
  • Readability10%

    7.5/10
  • Editorial score

    80.3/100

How the rank was computed

Rank No. 27 of 500 comes from a composite of 73.5: observable signals 37.3 of 55 points plus editorial review 36.2 of 45 points.

All 11 inputs
InputPoints
Observable signals37.3/55
Reach (Amazon ratings volume)10.8/16.5
Consensus across best-of lists13.8/13.8
Awards and bestseller evidence5.0/8.3
Reader rating, sample-adjusted6.1/8.3
Amazon sales rank1.6/5.5
Recency0.0/2.8
Editorial review36.2/45
Author authority10.7/11.3
Influence11.5/13.5
Practical value5.9/9.0
Readability3.4/4.5
Currency4.7/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

The most rigorous general-audience case that AI must be built to defer to uncertain human preferences, written by a field-defining researcher.

Strengths

  • Written by a foremost AI researcher
  • Concrete technical proposal, not only warnings
  • Engages skeptics' arguments point by point

Keep in mind

Published before the ChatGPT era, it says little about how today's chatbots are trained, and skeptics of superintelligence, including some reviewers, reject its premises.

Key takeaways

  1. Building machines that optimize fixed objectives is risky, because any mis-specified goal pursued by a capable system can produce harmful side effects.
  2. Beneficial machines should be uncertain about human preferences, which gives them an incentive to defer, ask permission and accept shutdown.
  3. Inverse reinforcement learning, which infers goals from observed behavior, offers one route for machines to learn what people actually want.
  4. Near-term misuse, including mass surveillance, behavioral manipulation and lethal autonomous weapons, demands attention alongside long-term control risks.

Notable facts

About the author

Stuart Russell

Stuart Russell is Distinguished Professor of Computer Science and Smith-Zadeh Professor in Engineering at UC Berkeley, where he leads the Center for Human-Compatible AI. With Peter Norvig he wrote Artificial Intelligence: A Modern Approach, adopted by more than 1,500 schools. He is an OBE and a Fellow of the Royal Society, and he delivered the BBC's 2021 Reith Lectures, Living With Artificial Intelligence.

Questions

What is Human Compatible about?
Human Compatible is Stuart Russell's argument that the standard way of building AI, giving machines fixed objectives to optimize, becomes dangerous as systems grow more capable. He proposes instead that machines pursue human preferences while remaining uncertain about what those preferences are, so they defer to people and accept correction. The book also covers AI misuse, the economics of automation and rebuttals to skeptics.
Is Human Compatible still relevant in the age of ChatGPT?
Yes. Although it was published in 2019, before ChatGPT brought large language models to a mass audience, its central problem of specifying what we want from powerful systems is now a practical concern for every AI lab. Ideas it popularized, such as learning preferences from human behavior and building machines that accept shutdown, remain central to current alignment research and policy debate.
Do I need a technical background to read Human Compatible?
No. Russell wrote the main text for general readers and explains concepts such as reinforcement learning and probability with everyday examples. Some passages on game theory and preference learning require careful reading, and optional appendices on search, logic, probability and learning give technically minded readers more depth without being necessary to follow the argument.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.