No. 407 of 500 No. 48 in Generative AI & LLMs

Designing Large Language Model Applications

A Holistic Approach to LLMs

By Suhas Pai

Print length
364 pages
Publisher
O'Reilly Media
Reading level
Advanced
ISBN-13
978-1098150501
Published
April 15, 2025
Written for
ML engineers and technical leads designing production LLM systems who want to understand model internals.

Synopsis

Suhas Pai argues that building dependable products on large language models requires understanding the models themselves, not only their APIs. The first part examines what goes into an LLM: pre-training datasets and their quirks, vocabulary and tokenization, and the Transformer architecture together with the learning objectives used to train it. The second part covers adapting models to real use cases, moving through model selection and prompting, fine-tuning, advanced fine-tuning methods, alignment training and reasoning, and techniques for making inference faster and cheaper.

The third part addresses application paradigms: connecting models to external tools and agents, representation learning and embeddings, retrieval-augmented generation, and a closing chapter on design patterns and system architecture. Throughout, Pai draws on the research literature and on his experience as CTO of an applied NLP startup, flagging failure modes that commonly appear in production and explaining why they occur. The result reads more like a compact graduate survey than a cookbook. Readers gain the vocabulary and intuition to evaluate new models, decide when fine-tuning is worth its cost compared with retrieval, and design systems that account for an LLM's limits, though they should arrive comfortable with Python and basic machine learning.

Scorecard

Editorial score

How it scored

Editorial score: 70.3 / 1005 weighted criteria

  • Influence30%

    18.0/30
  • Authority25%

    18.8/25
  • Practical value20%

    15.0/20
  • Currency15%

    12.0/15
  • Readability10%

    6.5/10
  • Editorial score

    70.3/100

How the rank was computed

Rank No. 407 of 500 comes from a composite of 49.5: observable signals 17.9 of 55 points plus editorial review 31.6 of 45 points.

All 11 inputs
InputPoints
Observable signals17.9/55
Reach (Amazon ratings volume)4.5/16.5
Consensus across best-of lists5.5/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted4.8/8.3
Amazon sales rank0.3/5.5
Recency2.8/2.8
Editorial review31.6/45
Author authority8.4/11.3
Influence8.1/13.5
Practical value6.8/9.0
Readability2.9/4.5
Currency5.4/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A rigorous, research-informed guide to how LLMs work and how to adapt them, written for engineers who want depth.

Strengths

  • Explains model internals relevant to application design
  • Covers fine-tuning, alignment, inference and RAG
  • Grounded in current research literature

Keep in mind

Dense and wide-ranging; readers without Python and machine learning grounding will find it demanding, and fast-moving topics may date quickly.

Key takeaways

  1. The composition and cleaning of pre-training data shape a model's strengths, blind spots and biases as much as its architecture does.
  2. Tokenization affects cost, context limits and performance on numbers and non-English text, so application builders need to understand it as well as model trainers.
  3. Fine-tuning, parameter-efficient methods and alignment training solve different problems, and the right choice depends on available data, budget and the behavior required.
  4. Retrieval-augmented generation and tool use extend a model beyond its training data, but each adds failure points that system design must anticipate.

Notable facts

About the author

Suhas Pai

Suhas Pai is co-founder and CTO of Hudson Labs, a Toronto-based, Y Combinator-backed applied NLP company. He has contributed to several open-source language models and co-led the Privacy working group of the BigScience project that produced the BLOOM model. He has chaired the Toronto Machine Learning Summit since 2021 and regularly hosts seminars on current NLP research.

Questions

Is Designing Large Language Model Applications worth reading in 2026?
Yes, for engineers who want lasting understanding rather than framework tutorials. Published in 2025, it covers pre-training data, tokenization, Transformer architectures, fine-tuning, alignment and reasoning, inference optimization, embeddings, RAG and agents. Because it emphasizes why techniques work and fail, much of it should outlast specific model releases, although agent frameworks and reasoning models continue to change faster than any print book can track.
Who should read Designing Large Language Model Applications?
It suits machine learning engineers, data scientists and technical leads who build or evaluate LLM-based products and are comfortable with Python and core machine learning ideas. Readers who want a quick prompt-engineering guide or a no-code overview will find it too technical. Product and engineering managers can read selected chapters to understand the trade-offs between fine-tuning, retrieval and model choice.
How is Designing Large Language Model Applications different from other LLM books?
Many LLM books focus on calling APIs or using a single framework. Suhas Pai starts with the ingredients of a model, including training data, tokenization and architecture, and connects them to practical decisions such as when to fine-tune, how to optimize inference and how to design retrieval and agent systems. The result is a holistic survey grounded in published research.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.