No. 113 of 500 No. 8 in Hands-On AI & Programming

Designing Machine Learning Systems

An Iterative Process for Production-Ready Applications

By Chip Huyen

Published
June 21, 2022
Print length
386 pages
Reading level
Intermediate
ISBN-13
978-1098107963
Publisher
O'Reilly Media
Written for
ML engineers, data scientists and engineering managers responsible for deploying models in production.

Synopsis

Chip Huyen developed this book from the machine learning systems design course she taught at Stanford, and it treats a production ML system as far more than a model: it includes data pipelines, features, infrastructure, monitoring and the people who maintain them. Early chapters contrast research ML with production ML, then work through data engineering fundamentals, training data and labeling, feature engineering, and model development and offline evaluation. Each decision, such as how to sample data or which metric to optimize, is framed by its effect on the system as a whole.

Later chapters address deployment and prediction services, why models fail in production, data distribution shifts and how to detect them, continual learning and testing in production, infrastructure and tooling for MLOps, and the human side of machine learning, including team structure and responsible AI. Huyen includes no code tutorials; instead she draws on case studies, research papers and industry practice to explain trade-offs. Readers gain a mental model for MLOps that outlasts specific tools, though the book predates the large language model wave, which her 2025 follow-up, AI Engineering, covers.

Scorecard

Editorial score

How it scored

Editorial score: 82.3 / 1005 weighted criteria

  • Influence30%

    25.5/30
  • Authority25%

    21.3/25
  • Practical value20%

    17.0/20
  • Currency15%

    10.5/15
  • Readability10%

    8.0/10
  • Editorial score

    82.3/100

How the rank was computed

Rank No. 113 of 500 comes from a composite of 62.2: observable signals 25.1 of 55 points plus editorial review 37.1 of 45 points.

All 11 inputs
InputPoints
Observable signals25.1/55
Reach (Amazon ratings volume)11.0/16.5
Consensus across best-of lists5.5/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted6.8/8.3
Amazon sales rank1.8/5.5
Recency0.0/2.8
Editorial review37.1/45
Author authority9.6/11.3
Influence11.5/13.5
Practical value7.7/9.0
Readability3.6/4.5
Currency4.7/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A clear, systems-level guide to taking machine learning from prototype to production, grounded in real engineering trade-offs rather than code recipes.

Strengths

  • Holistic view of the full ML lifecycle
  • Grounded in real industry case studies
  • Tool-agnostic principles that age well

Keep in mind

It contains no code and little mathematical depth, and it predates generative AI, so readers building LLM applications should pair it with newer material.

Key takeaways

  1. Production ML differs from research ML: data changes constantly, latency and cost matter, and success is measured against business objectives rather than leaderboard accuracy.
  2. Many failures in deployed models come from data problems such as distribution shifts, so monitoring inputs and outputs matters as much as model selection.
  3. Continual learning and testing in production, through methods such as shadow deployment, A/B tests and bandits, keep models useful as conditions change.
  4. Teams should invest in shared ML platforms and tooling that serve many use cases, weighing build-versus-buy decisions against their organization's maturity.

Notable facts

About the author

Chip Huyen

Chip Huyen is a computer scientist and writer on AI systems design. She taught CS 329S, Machine Learning Systems Design, at Stanford; worked on ML tooling at NVIDIA, where she was a core developer of NeMo, as well as at Snorkel AI and Netflix; and co-founded the AI infrastructure startup Claypot AI. Her later book, AI Engineering (2025), covers building applications on foundation models.

Questions

Is Designing Machine Learning Systems still relevant in 2026?
Largely, yes. Its principles on data quality, distribution shift, monitoring, deployment trade-offs and continual learning apply to any model in production, including systems built on foundation models. It does not cover prompt engineering, retrieval-augmented generation or LLM evaluation, which Chip Huyen treats in her 2025 follow-up, AI Engineering. Readers building LLM applications can treat the two books as a pair.
Does Designing Machine Learning Systems include code?
No. Huyen states that it is not a tutorial book, and it contains no code examples. It explains design decisions, trade-offs and patterns, citing research papers and industry case studies, so readers can apply its ideas with whatever stack they use. Readers who want step-by-step implementations in a specific framework should pair it with a hands-on programming text.
Who should read Designing Machine Learning Systems?
It is written for ML engineers, data scientists, data engineers, ML platform engineers and engineering managers, especially at mid-size and large companies and fast-growing startups. Huyen notes that readers without a strong technical background may find the first two chapters and the final chapter, on the human side of machine learning, the most useful starting points.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.