No. 218 of 500 No. 27 in Hands-On AI & Programming

Deep Reinforcement Learning Hands-On

By Maxim Lapan

Print length
716 pages
Publisher
Packt Publishing
Reading level
Intermediate
ISBN-13
978-1835882702
Published
November 12, 2024
Written for
Python developers and machine learning engineers who want to implement reinforcement learning algorithms themselves.

Synopsis

Maxim Lapan teaches reinforcement learning, the branch of machine learning in which an agent learns from trial, error and reward, almost entirely through working code. First published by Packt in 2018 and revised in 2020 and 2024, the book begins with agents, environments and rewards, the Gym toolkit and a short course in PyTorch, then builds from the cross-entropy method and tabular learning with the Bellman equation to deep Q-networks and their extensions. Policy gradients, actor-critic methods such as A2C and A3C, and trust-region methods including PPO, TRPO and SAC follow, along with black-box optimization and advanced exploration.

The 716-page third edition spans 22 chapters and applies these methods to stock trading, text adventure games in Microsoft's TextWorld, web navigation, continuous control, discrete optimization and multi-agent settings. New chapters cover reinforcement learning from human feedback, the technique widely used to fine-tune large language models, and MuZero, DeepMind's model-based successor to AlphaZero. Methods are tied to the research papers that introduced them and implemented with Gymnasium, PyTorch Ignite and the PTAN helper library. Readers who work through the code can implement the core algorithms behind modern RL papers and see firsthand why training is often unstable and which techniques stabilize it.

Scorecard

Editorial score

How it scored

Editorial score: 71.8 / 1005 weighted criteria

  • Influence30%

    19.5/30
  • Authority25%

    16.3/25
  • Practical value20%

    17.0/20
  • Currency15%

    12.0/15
  • Readability10%

    7.0/10
  • Editorial score

    71.8/100

How the rank was computed

Rank No. 218 of 500 comes from a composite of 56.0: observable signals 23.6 of 55 points plus editorial review 32.4 of 45 points.

All 11 inputs
InputPoints
Observable signals23.6/55
Reach (Amazon ratings volume)7.0/16.5
Consensus across best-of lists8.3/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted5.7/8.3
Amazon sales rank0.9/5.5
Recency1.7/2.8
Editorial review32.4/45
Author authority7.3/11.3
Influence8.8/13.5
Practical value7.7/9.0
Readability3.2/4.5
Currency5.4/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A thorough, code-first PyTorch guide to deep reinforcement learning, from Q-learning to PPO, RLHF and MuZero in its 2024 edition.

Strengths

  • Runnable PyTorch code for nearly every method
  • Wide range of applied environments and domains
  • Updated for RLHF, MuZero and Gymnasium

Keep in mind

Readers need solid Python, calculus and machine learning basics; theory is treated briefly, so those wanting rigorous derivations should pair it with Sutton and Barto.

Key takeaways

  1. Simple methods such as the cross-entropy method can solve small environments and build intuition before moving to deep Q-networks and policy gradients.
  2. DQN extensions such as double DQN, dueling networks and prioritized replay each address a specific failure mode, and several can be combined.
  3. Policy-gradient and actor-critic methods such as A2C, PPO and SAC handle continuous action spaces where value-based methods struggle.
  4. Reinforcement learning from human feedback trains a reward model on human preferences, the same idea used to fine-tune modern large language models.

Notable facts

  • The third edition, published by Packt on November 12, 2024, runs to 716 pages and adds material on RLHF and MuZero. Source (opens in a new tab)
  • The third edition's companion repository spans 22 chapters, with code tested on Python 3.11, PyTorch 2.5 and Gymnasium. Source (opens in a new tab)

About the author

Maxim Lapan

Maxim Lapan has worked as a software developer for more than two decades in distributed scientific computing, distributed systems and big data processing. Since 2014 he has applied machine learning and deep learning to industrial problems, including NLP and reinforcement learning for web crawling and web page analysis. He has written three editions of Deep Reinforcement Learning Hands-On for Packt and lives in Germany.

Questions

Which edition of Deep Reinforcement Learning Hands-On should I read?
Choose the third edition, published by Packt in November 2024. Its code targets Gymnasium and recent PyTorch releases, and it includes chapters on reinforcement learning from human feedback and MuZero. The 2018 and 2020 editions rely on older library versions, so their code may need changes before it runs, and they lack the newer material.
Do I need to know PyTorch before reading Deep Reinforcement Learning Hands-On?
No prior PyTorch experience is required, because an early chapter introduces the library, but you should be comfortable with Python, basic calculus and core machine learning ideas. A CUDA-capable GPU, a cloud instance or Google Colab will make the larger experiments, such as Atari games, run considerably faster.
Is Deep Reinforcement Learning Hands-On worth reading in 2026?
Yes, for engineers who learn by implementing. The 2024 edition covers algorithms still used in practice, including DQN variants, PPO, SAC and RLHF, with runnable code. It does not survey the newest reinforcement learning methods for training reasoning models, so readers focused on that area should add recent papers or courses.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.