No. 18 of 500 No. 5 in AI Safety & the Future

Superintelligence

Paths, Dangers, Strategies

By Nick Bostrom

Published
May 1, 2016
Print length
432 pages
Reading level
Advanced
ISBN-13
978-0198739838
Publisher
Oxford University Press
Written for
Researchers, policymakers and committed readers who want the foundational case for taking AI risk seriously.

Synopsis

Nick Bostrom asks what happens if machines come to exceed human intelligence across virtually every domain. He surveys possible routes, including artificial intelligence, whole brain emulation, biological enhancement and networked organizations, and argues that once human-level machine intelligence exists, a rapid “takeoff” to superintelligence is plausible. Two ideas carry much of the argument: the orthogonality thesis, which holds that intelligence and final goals vary independently, so a highly capable system need not share human values; and instrumental convergence, which predicts that almost any goal would lead such a system to seek resources, self-preservation and protection of its objectives.

The later chapters address the control problem: capability-control methods such as boxing and tripwires, motivation-selection methods such as indirect normativity, and the strategic question of whether one project or many would reach superintelligence first. Bostrom writes as an analytic philosopher, carefully qualifying claims, which makes the book slow going but precise. Written before the transformer era, it says little about today's language models, yet concepts it popularized, including the treacherous turn and perverse instantiation, are standard vocabulary in AI safety research. Readers gain the conceptual foundations behind current alignment and governance debates.

Scorecard

Editorial score

How it scored

Editorial score: 73.3 / 1005 weighted criteria

  • Influence30%

    27.0/30
  • Authority25%

    21.3/25
  • Practical value20%

    10.0/20
  • Currency15%

    10.5/15
  • Readability10%

    4.5/10
  • Editorial score

    73.3/100

How the rank was computed

Rank No. 18 of 500 comes from a composite of 75.5: observable signals 42.5 of 55 points plus editorial review 33.0 of 45 points.

All 11 inputs
InputPoints
Observable signals42.5/55
Reach (Amazon ratings volume)13.5/16.5
Consensus across best-of lists13.8/13.8
Awards and bestseller evidence8.3/8.3
Reader rating, sample-adjusted4.8/8.3
Amazon sales rank2.1/5.5
Recency0.0/2.8
Editorial review33.0/45
Author authority9.6/11.3
Influence12.2/13.5
Practical value4.5/9.0
Readability2.0/4.5
Currency4.7/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

The rigorous, demanding book that framed the modern AI-risk debate and supplied much of the alignment field's vocabulary.

Strengths

  • Systematic taxonomy of risks and control methods
  • Introduced durable concepts still used in alignment
  • Careful, qualified philosophical reasoning throughout

Keep in mind

Dense, abstract prose makes it hard going for general readers, and its speculative scenarios predate the language-model systems that dominate today's AI.

Key takeaways

  1. Intelligence and goals are independent, so a superintelligent system could pursue objectives indifferent or hostile to human welfare without any malice.
  2. Almost any final goal creates incentives to acquire resources, resist shutdown and preserve the goal itself, so dangerous behavior can arise from harmless-sounding objectives.
  3. A system might behave cooperatively while weak and change course once it is strong enough, so good behavior during testing proves little.
  4. The control problem must be solved before superintelligence arrives, because humanity may get only one chance to set a powerful system's goals correctly.

Notable facts

  • Ranked No. 17 on The New York Times best-selling science books list in August 2014. Source (opens in a new tab)
  • Bill Gates said in 2015 that he would highly recommend the book, and Sam Altman wrote that it was the best thing he had read on AI risk. Source (opens in a new tab)

About the author

Nick Bostrom

Nick Bostrom was a professor at the University of Oxford, where he was founding director of the Future of Humanity Institute from 2005 until its closure in 2024. He now leads the nonprofit Macrostrategy Research Initiative as founder and principal researcher. His background spans philosophy, theoretical physics and computational neuroscience, and his books include Anthropic Bias and Deep Utopia (2024).

Questions

What is Superintelligence about?
Nick Bostrom's 2014 book analyzes how machines might come to surpass human intelligence and why that could be dangerous. He argues that a superintelligent system with poorly specified goals could pursue them in ways catastrophic for people, and he surveys methods for controlling such systems or shaping their motivations before they are built.
Is Superintelligence worth reading in 2026?
Yes, for readers who want the most influential book-length treatment of ideas that now shape AI safety. Concepts such as the orthogonality thesis, instrumental convergence and the treacherous turn, which the book brought to wide attention, inform how AI labs and researchers discuss alignment today. Its scenarios predate large language models, so pair it with recent work on how current systems are trained and evaluated.
Is Superintelligence hard to read?
It is demanding. Bostrom writes in the careful, qualified style of academic philosophy, with technical terms, boxed digressions and long chains of argument. Reviewers including The Telegraph described it as a hard but rewarding read. Readers new to the subject may want an accessible introduction first, then return to this book for its depth.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.