No. 311 of 500 No. 61 in Machine Learning

Understanding Machine Learning

From Theory to Algorithms

By Shai Shalev-Shwartz and Shai Ben-David

Published
July 17, 2014
Print length
414 pages
Reading level
Advanced
ISBN-13
978-1107057135
Publisher
Cambridge University Press
Written for
Graduate students, researchers and mathematically minded engineers who want the theory behind machine learning.

Synopsis

Shai Shalev-Shwartz and Shai Ben-David set out to explain machine learning from first principles: what it means for an algorithm to learn, how much data learning requires, and when learning can be done efficiently. The book opens with the formal PAC learning model, uniform convergence, the bias-complexity trade-off and the VC dimension, then shows how those ideas yield concrete algorithms. Later parts treat linear predictors, boosting, convex learning and stochastic gradient descent, support vector machines and kernels, decision trees, nearest neighbors and neural networks, followed by online learning, clustering, dimensionality reduction, generative models and feature selection.

A closing part covers advanced theory, including Rademacher complexity, covering numbers, compression bounds and PAC-Bayes analysis. The treatment is mathematical throughout and contains no code; proofs are complete, and exercises accompany the chapters. Readers who work through it gain a precise vocabulary for overfitting, sample complexity and the limits of learning, along with the tools to read theoretical research papers. Because the authors post a free PDF for personal use, the text is a common choice for graduate courses in learning theory. Its coverage of deep learning is brief, reflecting the field in 2014, so practitioners seeking modern architectures will need other sources.

Scorecard

Editorial score

How it scored

Editorial score: 76.5 / 1005 weighted criteria

  • Influence30%

    25.5/30
  • Authority25%

    22.5/25
  • Practical value20%

    12.0/20
  • Currency15%

    10.5/15
  • Readability10%

    6.0/10
  • Editorial score

    76.5/100

How the rank was computed

Rank No. 311 of 500 comes from a composite of 52.6: observable signals 18.2 of 55 points plus editorial review 34.4 of 45 points.

All 11 inputs
InputPoints
Observable signals18.2/55
Reach (Amazon ratings volume)9.2/16.5
Consensus across best-of lists2.8/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted5.4/8.3
Amazon sales rank0.8/5.5
Recency0.0/2.8
Editorial review34.4/45
Author authority10.1/11.3
Influence11.5/13.5
Practical value5.4/9.0
Readability2.7/4.5
Currency4.7/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A rigorous, elegantly organized introduction to learning theory that explains why algorithms generalize; demanding mathematics, but freely available and widely taught.

Strengths

  • Rigorous, self-contained proofs throughout
  • Clear progression from theory to algorithms
  • Free PDF available from the authors

Keep in mind

Readers without comfort in probability, linear algebra and proofs will struggle, and deep learning theory developed after 2014 is largely absent.

Key takeaways

  1. The PAC framework formalizes learning as finding, with high probability, a hypothesis whose error is close to the best achievable within a class.
  2. The VC dimension characterizes which hypothesis classes are learnable and how many examples learning requires, linking model capacity to sample size.
  3. The bias-complexity trade-off explains overfitting: richer hypothesis classes reduce approximation error but demand more data to control estimation error.
  4. Convexity, stability and regularization explain why methods such as stochastic gradient descent and support vector machines generalize from finite samples.

Notable facts

  • The University of Arizona's CSC 665 Machine Learning Theory course lists it as the required textbook, noting the authors make it available online for personal use. Source (opens in a new tab)
  • The authors distribute the full book as a free PDF from their website, alongside the Cambridge University Press print edition. Source (opens in a new tab)

About the author

Shai Shalev-Shwartz and Shai Ben-David

Shai Shalev-Shwartz is a professor of computer science at the Hebrew University of Jerusalem and chief technology officer of Mobileye, where he co-developed the Responsibility-Sensitive Safety framework for automated driving; he has written more than 200 research papers. Co-author Shai Ben-David has been a computer science professor at the University of Waterloo since 2004, previously taught at the Technion, and is an ACM Fellow.

Questions

Is Understanding Machine Learning worth reading in 2026?
Yes, for anyone who wants to understand why learning algorithms work. The core results on PAC learning, VC dimension, regularization and stability have not changed, and they underpin current research on generalization. The book predates the transformer era and treats neural networks briefly, so readers should pair it with a modern deep learning text for architectures and training practice.
Who should read Understanding Machine Learning?
It is aimed at advanced undergraduates and beginning graduate students in computer science, statistics, mathematics and engineering, and at researchers who need a rigorous foundation. Readers should be comfortable with probability, linear algebra and mathematical proofs. Practitioners seeking code, libraries or deployment advice will find none here and should choose a hands-on programming book instead.
What is Understanding Machine Learning about?
Shai Shalev-Shwartz and Shai Ben-David present machine learning as a mathematical discipline. The book defines learnability through the PAC model, characterizes it with the VC dimension, derives algorithms such as boosting, support vector machines and stochastic gradient descent from those principles, and closes with advanced tools including Rademacher complexity and PAC-Bayes bounds. A free PDF is available from the authors.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.