Learning From Data
A Short Course
By Yaser S. Abu-Mostafa, Malik Magdon-Ismail and Hsuan-Tien Lin
Synopsis
Learning From Data grew out of more than a decade of teaching by Yaser Abu-Mostafa at Caltech, Malik Magdon-Ismail at Rensselaer Polytechnic Institute and Hsuan-Tien Lin at National Taiwan University, and it serves as the main text for their machine learning courses. Rather than surveying many algorithms, it concentrates on one question: when can a model trained on a finite sample be trusted on data it has never seen? Five chapters build the answer step by step, starting with the perceptron and the feasibility of learning, moving to the Vapnik-Chervonenkis dimension, generalization bounds and the bias-variance tradeoff, then linear and logistic regression with nonlinear transforms, then overfitting, regularization and validation, and closing with principles such as Occam's razor, sampling bias and data snooping.
The authors write in a narrative sequence, with more than 50 color illustrations and over 100 exercises and problems, and they balance mathematical results with heuristics that matter in practice. Free online e-Chapters extend the material to neural networks, support vector machines and similarity-based methods, and Abu-Mostafa's recorded Caltech lectures follow the text closely. Readers gain the conceptual tools to diagnose overfitting and to judge whether a model's reported performance will hold up on new data. Readers expecting deep learning, Python code or a broad catalog of algorithms will need a companion text.
Scorecard
Editorial score
How it scored
Editorial score: 76.1 / 1005 weighted criteria
- 24.0/30
Influence30%
- 21.3/25
Authority25%
- 13.0/20
Practical value20%
- 9.8/15
Currency15%
- 8.0/10
Readability10%
- 76.1/100
Editorial score
How the rank was computed
Rank No. 294 of 500 comes from a composite of 53.3: observable signals 19.0 of 55 points plus editorial review 34.3 of 45 points.
All 11 inputs
| Input | Points |
|---|---|
| Observable signals | 19.0/55 |
| Reach (Amazon ratings volume) | 9.0/16.5 |
| Consensus across best-of lists | 2.8/13.8 |
| Awards and bestseller evidence | 0.0/8.3 |
| Reader rating, sample-adjusted | 6.6/8.3 |
| Amazon sales rank | 0.6/5.5 |
| Recency | 0.0/2.8 |
| Editorial review | 34.3/45 |
| Author authority | 9.6/11.3 |
| Influence | 10.8/13.5 |
| Practical value | 5.9/9.0 |
| Readability | 3.6/4.5 |
| Currency | 4.4/6.8 |
Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.
Verdict
A compact, carefully reasoned introduction to why machine learning works, built around generalization theory rather than algorithms or code.
Strengths
- Exceptionally clear treatment of generalization theory
- Paired with free Caltech video lectures
- Well-chosen exercises reinforce every key concept
Keep in mind
Its narrow scope omits deep learning, tree ensembles and coding practice, and some readers find the mathematics more theoretical than they expected.
Key takeaways
- Learning is feasible only in a probabilistic sense: with enough data, in-sample performance can track out-of-sample performance within quantifiable bounds.
- The VC dimension measures a model's effective capacity and determines how much data is needed for reliable generalization.
- Overfitting arises from both stochastic noise and deterministic noise, the part of the target function a model is too simple to capture.
- Data snooping, looking at the data before choosing a model, quietly invalidates performance estimates, and sampling bias can undermine learning entirely.
Notable facts
- Caltech's free Learning From Data online course, taught by Abu-Mostafa, reports more than 9 million views on YouTube and other servers. Source (opens in a new tab)
- The book is the main text for the authors' machine learning courses at Caltech, RPI and National Taiwan University. Source (opens in a new tab)
- Lead author Yaser Abu-Mostafa received Caltech's 1996 Richard P. Feynman Prize for Excellence in Teaching. Source (opens in a new tab)
About the author
Questions
Is Learning From Data good for beginners?
Does Learning From Data cover deep learning?
Is there a course that goes with Learning From Data?
Related books
- Causal Inference Scott Cunningham
- High-Dimensional Statistics Martin J. Wainwright
- Multi-Agent Reinforcement Learning Stefano V. Albrecht, Filippos Christianos and Lukas Schäfer
- Machine Learning Tom M. Mitchell
- Machine Learning Q and AI Sebastian Raschka
- Understanding Machine Learning Shai Shalev-Shwartz and Shai Ben-David
Sources
As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.






