Deep Reinforcement Learning Hands-On
By Maxim Lapan
Synopsis
Maxim Lapan teaches reinforcement learning, the branch of machine learning in which an agent learns from trial, error and reward, almost entirely through working code. First published by Packt in 2018 and revised in 2020 and 2024, the book begins with agents, environments and rewards, the Gym toolkit and a short course in PyTorch, then builds from the cross-entropy method and tabular learning with the Bellman equation to deep Q-networks and their extensions. Policy gradients, actor-critic methods such as A2C and A3C, and trust-region methods including PPO, TRPO and SAC follow, along with black-box optimization and advanced exploration.
The 716-page third edition spans 22 chapters and applies these methods to stock trading, text adventure games in Microsoft's TextWorld, web navigation, continuous control, discrete optimization and multi-agent settings. New chapters cover reinforcement learning from human feedback, the technique widely used to fine-tune large language models, and MuZero, DeepMind's model-based successor to AlphaZero. Methods are tied to the research papers that introduced them and implemented with Gymnasium, PyTorch Ignite and the PTAN helper library. Readers who work through the code can implement the core algorithms behind modern RL papers and see firsthand why training is often unstable and which techniques stabilize it.
Scorecard
Editorial score
How it scored
Editorial score: 71.8 / 1005 weighted criteria
- 19.5/30
Influence30%
- 16.3/25
Authority25%
- 17.0/20
Practical value20%
- 12.0/15
Currency15%
- 7.0/10
Readability10%
- 71.8/100
Editorial score
How the rank was computed
Rank No. 218 of 500 comes from a composite of 56.0: observable signals 23.6 of 55 points plus editorial review 32.4 of 45 points.
All 11 inputs
| Input | Points |
|---|---|
| Observable signals | 23.6/55 |
| Reach (Amazon ratings volume) | 7.0/16.5 |
| Consensus across best-of lists | 8.3/13.8 |
| Awards and bestseller evidence | 0.0/8.3 |
| Reader rating, sample-adjusted | 5.7/8.3 |
| Amazon sales rank | 0.9/5.5 |
| Recency | 1.7/2.8 |
| Editorial review | 32.4/45 |
| Author authority | 7.3/11.3 |
| Influence | 8.8/13.5 |
| Practical value | 7.7/9.0 |
| Readability | 3.2/4.5 |
| Currency | 5.4/6.8 |
Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.
Verdict
A thorough, code-first PyTorch guide to deep reinforcement learning, from Q-learning to PPO, RLHF and MuZero in its 2024 edition.
Strengths
- Runnable PyTorch code for nearly every method
- Wide range of applied environments and domains
- Updated for RLHF, MuZero and Gymnasium
Keep in mind
Readers need solid Python, calculus and machine learning basics; theory is treated briefly, so those wanting rigorous derivations should pair it with Sutton and Barto.
Key takeaways
- Simple methods such as the cross-entropy method can solve small environments and build intuition before moving to deep Q-networks and policy gradients.
- DQN extensions such as double DQN, dueling networks and prioritized replay each address a specific failure mode, and several can be combined.
- Policy-gradient and actor-critic methods such as A2C, PPO and SAC handle continuous action spaces where value-based methods struggle.
- Reinforcement learning from human feedback trains a reward model on human preferences, the same idea used to fine-tune modern large language models.
Notable facts
- The third edition, published by Packt on November 12, 2024, runs to 716 pages and adds material on RLHF and MuZero. Source (opens in a new tab)
- The third edition's companion repository spans 22 chapters, with code tested on Python 3.11, PyTorch 2.5 and Gymnasium. Source (opens in a new tab)
About the author
Questions
Which edition of Deep Reinforcement Learning Hands-On should I read?
Do I need to know PyTorch before reading Deep Reinforcement Learning Hands-On?
Is Deep Reinforcement Learning Hands-On worth reading in 2026?
Related books
- Python Data Science Handbook Jake VanderPlas
- Fundamentals of Data Engineering Joe Reis and Matt Housley
- Machine Learning Design Patterns Valliappa Lakshmanan, Sara Robinson and Michael Munn
- Machine Learning with PyTorch and Scikit-Learn Sebastian Raschka, Yuxi (Hayden) Liu and Vahid Mirjalili
- Introduction to Machine Learning with Python Andreas C. Müller and Sarah Guido
- Data Science from Scratch Joel Grus
Sources
As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.






