No. 162 of 500 No. 16 in Generative AI & LLMs

The Hundred-Page Language Models Book

By Andriy Burkov

Published
January 15, 2025
Print length
156 pages
Reading level
Intermediate
ISBN-13
978-1778042720
Publisher
True Positive Inc.
Written for
Software engineers and data scientists who want to understand and code language models from scratch.

Synopsis

Andriy Burkov applies the format of his earlier Hundred-Page Machine Learning Book to language models, compressing the route from first principles to modern large language models into about 160 pages. A fast review of machine learning basics, covering vectors, gradient descent and neural network training, leads into language modeling fundamentals: bag of words, word embeddings, byte-pair encoding, count-based models and evaluation by perplexity. Successive chapters implement a recurrent neural network language model and then a Transformer from scratch in PyTorch, before the longest chapter turns to large language models, including instruction finetuning, low-rank adaptation (LoRA), prompt engineering and getting consistent outputs from a model.

Each idea is presented three ways, in mathematical notation, in diagrams and in Python code, with companion notebooks published on GitHub. The result reads like a dense course reader rather than a narrative, and it rewards readers who run the code as they go. Readers come away able to explain how next-token prediction, attention and finetuning work and to train small models themselves, which makes later work with production LLMs less opaque. Burkov sells the book on a read-first, buy-later basis, posting chapters free on the book's website, and word2vec creator Tomáš Mikolov wrote the foreword.

Scorecard

Editorial score

How it scored

Editorial score: 71.8 / 1005 weighted criteria

  • Influence30%

    18.0/30
  • Authority25%

    17.5/25
  • Practical value20%

    16.0/20
  • Currency15%

    12.8/15
  • Readability10%

    7.5/10
  • Editorial score

    71.8/100

How the rank was computed

Rank No. 162 of 500 comes from a composite of 58.7: observable signals 26.4 of 55 points plus editorial review 32.3 of 45 points.

All 11 inputs
InputPoints
Observable signals26.4/55
Reach (Amazon ratings volume)7.5/16.5
Consensus across best-of lists8.3/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted6.7/8.3
Amazon sales rank1.1/5.5
Recency2.8/2.8
Editorial review32.3/45
Author authority7.9/11.3
Influence8.1/13.5
Practical value7.2/9.0
Readability3.4/4.5
Currency5.7/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A compact, code-driven path from count-based language models to transformers and finetuned LLMs, ideal for engineers who want the mechanics.

Strengths

  • Builds each architecture from scratch in PyTorch
  • Math, diagrams and code side by side
  • Chapters free to read before buying

Keep in mind

The compression is relentless: readers without working Python and basic linear algebra will find the pace steep, and there is little historical or narrative context.

Key takeaways

  1. Language modeling reduces to predicting the next token, and that single objective connects simple count-based models to billion-parameter transformers.
  2. Tokenization schemes such as byte-pair encoding and learned word embeddings turn raw text into numbers, and these choices shape what a model can learn.
  3. Self-attention lets a Transformer weigh every earlier token when predicting the next one, which is why it displaced recurrent networks for language tasks.
  4. Instruction finetuning and low-rank adaptation let practitioners adapt a pretrained model to new tasks without the cost of full retraining.

Notable facts

About the author

Andriy Burkov

Andriy Burkov holds a PhD in artificial intelligence and works as a machine learning team leader at TalentNeuron; he previously led production AI projects at Fujitsu and Gartner. He is the author of The Hundred-Page Machine Learning Book, now available in 12 languages, and Machine Learning Engineering, and his books are used as university textbooks.

Questions

What is The Hundred-Page Language Models Book about?
It explains how language models work, from count-based models and word embeddings through recurrent networks to the Transformer and modern large language models. Andriy Burkov pairs each concept with math, diagrams and PyTorch code, and the final chapter covers instruction finetuning, low-rank adaptation and prompt engineering, so readers both understand and build working models.
Do I need machine learning experience to read The Hundred-Page Language Models Book?
No formal experience is required, because the first chapter reviews machine learning basics such as gradient descent and neural network training. The pace is fast, however. Readers should be comfortable writing Python and reading basic linear algebra and calculus notation, or they will struggle to follow the derivations and code in the Transformer and LLM chapters.
Is The Hundred-Page Language Models Book worth reading in 2026?
Yes, for engineers who want the mechanics behind LLMs rather than a survey of products. Published in January 2025, it covers the Transformer and finetuning techniques that remain standard. Because chapters are free to read on thelmbook.com under a read-first, buy-later model, readers can sample the material before deciding to purchase a copy.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.