No. 151 of 500 No. 13 in Deep Learning

Natural Language Processing with Transformers

Building Language Applications with Hugging Face

By Lewis Tunstall, Leandro von Werra and Thomas Wolf

Published
July 5, 2022
Print length
406 pages
Reading level
Intermediate
ISBN-13
978-1098136796
Publisher
O'Reilly Media
Written for
Python developers and data scientists who want to build NLP applications with Hugging Face Transformers

Synopsis

Written by Lewis Tunstall, Leandro von Werra and Thomas Wolf, three contributors to the Hugging Face ecosystem, this book teaches transformer models through code built on the Transformers, Datasets and Tokenizers libraries. After an introduction to the encoder-decoder architecture, attention and transfer learning, each chapter centers on a realistic task: classifying the emotions expressed in tweets, recognizing named entities across several languages, generating text with different decoding strategies, summarizing dialogue, and building a question-answering system over product reviews. A chapter on transformer anatomy implements the architecture's components from scratch in PyTorch so that readers understand what the library abstracts.

Later chapters address problems practitioners meet after a first prototype. The authors show how to shrink and speed up models through knowledge distillation, quantization, pruning and ONNX export, how to cope with scarce labeled data using zero-shot and few-shot techniques, and how to train a code-generating model from scratch, including building a tokenizer and handling large-scale data. A closing chapter surveys scaling laws and multimodal directions as they stood in 2022. Companion Jupyter notebooks run on cloud GPUs. Readers gain practical fluency with the Hugging Face stack and a working understanding of why transformers succeed, though some of the code now needs updating.

Scorecard

Editorial score

How it scored

Editorial score: 77.5 / 1005 weighted criteria

  • Influence30%

    22.5/30
  • Authority25%

    22.5/25
  • Practical value20%

    16.0/20
  • Currency15%

    9.0/15
  • Readability10%

    7.5/10
  • Editorial score

    77.5/100

How the rank was computed

Rank No. 151 of 500 comes from a composite of 59.6: observable signals 24.7 of 55 points plus editorial review 34.9 of 45 points.

All 11 inputs
InputPoints
Observable signals24.7/55
Reach (Amazon ratings volume)8.9/16.5
Consensus across best-of lists8.3/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted6.6/8.3
Amazon sales rank0.9/5.5
Recency0.0/2.8
Editorial review34.9/45
Author authority10.1/11.3
Influence10.1/13.5
Practical value7.2/9.0
Readability3.4/4.5
Currency4.1/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

Hugging Face's own developers teach transformer NLP through clear, working projects, though the fast-moving library has outpaced parts of the code.

Strengths

  • Written by core Hugging Face library contributors
  • Task-driven chapters with runnable companion notebooks
  • Covers deployment efficiency, not only modeling

Keep in mind

Predates the instruction-tuned chat models that followed ChatGPT, and its repository notes untested TensorFlow code and an unmaintained question-answering chapter.

Key takeaways

  1. Transfer learning lets practitioners fine-tune a pretrained transformer for a new task with modest data and compute instead of training from scratch.
  2. Implementing self-attention and the encoder and decoder blocks by hand demystifies what high-level library calls are doing internally.
  3. Production deployment depends on efficiency: distillation, quantization and pruning can cut latency and model size with limited accuracy loss.
  4. When labels are scarce, zero-shot classification, data augmentation and domain adaptation can deliver usable models before investing in annotation.

Notable facts

About the author

Lewis Tunstall, Leandro von Werra and Thomas Wolf

Lewis Tunstall is a machine learning engineer at Hugging Face and a former theoretical physicist who has worked on post-training language models. Leandro von Werra, a Hugging Face research lead, created the TRL library and co-led the BigCode project behind StarCoder. Thomas Wolf co-founded Hugging Face, serves as its Chief Science Officer, helped build the Transformers and Datasets libraries and initiated the BigScience research collaboration.

Questions

What is Natural Language Processing with Transformers about?
Natural Language Processing with Transformers is a practical O'Reilly guide to building language applications with the Hugging Face Transformers library. It explains the transformer architecture, then works through text classification, multilingual named entity recognition, text generation, summarization and question answering, before covering model compression, learning with few labels and training a model from scratch.
Is Natural Language Processing with Transformers still worth reading in 2026?
For foundations, yes. Its explanations of attention, fine-tuning, tokenization and efficient deployment still apply to modern large language models. The book was published in 2022, however, so it does not cover the instruction-tuned chat models and alignment techniques that followed ChatGPT, and the companion repository notes that some code paths are no longer maintained or tested.
Who should read Natural Language Processing with Transformers?
Natural Language Processing with Transformers suits data scientists, machine learning engineers and Python developers who know basic deep learning and want to apply transformers. Familiarity with PyTorch helps, since the from-scratch chapters use it. Executives and general readers should look elsewhere, because the book is code-first, and researchers seeking mathematical depth may find its theory coverage deliberately brief.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.