No. 294 of 500 No. 56 in Machine Learning

Learning From Data

A Short Course

By Yaser S. Abu-Mostafa, Malik Magdon-Ismail and Hsuan-Tien Lin

Print length
213 pages
Publisher
AMLBook
Reading level
Intermediate
ISBN-13
978-1600490064
Published
January 1, 2012
Written for
Students and self-learners wanting a rigorous but approachable foundation in machine learning theory.

Synopsis

Learning From Data grew out of more than a decade of teaching by Yaser Abu-Mostafa at Caltech, Malik Magdon-Ismail at Rensselaer Polytechnic Institute and Hsuan-Tien Lin at National Taiwan University, and it serves as the main text for their machine learning courses. Rather than surveying many algorithms, it concentrates on one question: when can a model trained on a finite sample be trusted on data it has never seen? Five chapters build the answer step by step, starting with the perceptron and the feasibility of learning, moving to the Vapnik-Chervonenkis dimension, generalization bounds and the bias-variance tradeoff, then linear and logistic regression with nonlinear transforms, then overfitting, regularization and validation, and closing with principles such as Occam's razor, sampling bias and data snooping.

The authors write in a narrative sequence, with more than 50 color illustrations and over 100 exercises and problems, and they balance mathematical results with heuristics that matter in practice. Free online e-Chapters extend the material to neural networks, support vector machines and similarity-based methods, and Abu-Mostafa's recorded Caltech lectures follow the text closely. Readers gain the conceptual tools to diagnose overfitting and to judge whether a model's reported performance will hold up on new data. Readers expecting deep learning, Python code or a broad catalog of algorithms will need a companion text.

Scorecard

Editorial score

How it scored

Editorial score: 76.1 / 1005 weighted criteria

  • Influence30%

    24.0/30
  • Authority25%

    21.3/25
  • Practical value20%

    13.0/20
  • Currency15%

    9.8/15
  • Readability10%

    8.0/10
  • Editorial score

    76.1/100

How the rank was computed

Rank No. 294 of 500 comes from a composite of 53.3: observable signals 19.0 of 55 points plus editorial review 34.3 of 45 points.

All 11 inputs
InputPoints
Observable signals19.0/55
Reach (Amazon ratings volume)9.0/16.5
Consensus across best-of lists2.8/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted6.6/8.3
Amazon sales rank0.6/5.5
Recency0.0/2.8
Editorial review34.3/45
Author authority9.6/11.3
Influence10.8/13.5
Practical value5.9/9.0
Readability3.6/4.5
Currency4.4/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A compact, carefully reasoned introduction to why machine learning works, built around generalization theory rather than algorithms or code.

Strengths

  • Exceptionally clear treatment of generalization theory
  • Paired with free Caltech video lectures
  • Well-chosen exercises reinforce every key concept

Keep in mind

Its narrow scope omits deep learning, tree ensembles and coding practice, and some readers find the mathematics more theoretical than they expected.

Key takeaways

  1. Learning is feasible only in a probabilistic sense: with enough data, in-sample performance can track out-of-sample performance within quantifiable bounds.
  2. The VC dimension measures a model's effective capacity and determines how much data is needed for reliable generalization.
  3. Overfitting arises from both stochastic noise and deterministic noise, the part of the target function a model is too simple to capture.
  4. Data snooping, looking at the data before choosing a model, quietly invalidates performance estimates, and sampling bias can undermine learning entirely.

Notable facts

  • Caltech's free Learning From Data online course, taught by Abu-Mostafa, reports more than 9 million views on YouTube and other servers. Source (opens in a new tab)
  • The book is the main text for the authors' machine learning courses at Caltech, RPI and National Taiwan University. Source (opens in a new tab)
  • Lead author Yaser Abu-Mostafa received Caltech's 1996 Richard P. Feynman Prize for Excellence in Teaching. Source (opens in a new tab)

About the author

Yaser S. Abu-Mostafa, Malik Magdon-Ismail and Hsuan-Tien Lin

Yaser S. Abu-Mostafa is professor of electrical engineering and computer science at Caltech, where he won the 1996 Feynman Prize for Excellence in Teaching, and he cofounded the Neural Information Processing Systems conference in 1987. Malik Magdon-Ismail is a professor of computer science at Rensselaer Polytechnic Institute. Hsuan-Tien Lin is a professor of computer science and information engineering at National Taiwan University.

Questions

Is Learning From Data good for beginners?
It suits motivated beginners with college-level calculus, linear algebra and probability. The book starts from first principles and explains each result in plain language before formalizing it, which makes it one of the more approachable theory texts. Readers with no mathematical background, or those who want to start coding models immediately, may prefer a hands-on introduction first.
Does Learning From Data cover deep learning?
Not in the printed book. The five printed chapters focus on the feasibility of learning, generalization, linear models, overfitting, regularization and validation. The authors offer free online e-Chapters on neural networks, support vector machines and similarity-based methods, but modern architectures such as convolutional networks and transformers fall outside its scope, so readers need a separate deep learning text.
Is there a course that goes with Learning From Data?
Yes. Yaser Abu-Mostafa's Caltech course, recorded during a live broadcast in April and May 2012, has 18 lectures with homework sets and a final exam, and it is free online. Caltech reports more than 9 million views. Malik Magdon-Ismail also offers lecture slides and an online course at Rensselaer whose first part serves as a companion to the book.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.