An Illustrated Guide to AI Agents
Concepts and Code for Building Agents with LLMs, Tools, and Memory
Synopsis
Maarten Grootendorst and Jay Alammar, authors of Hands-On Large Language Models, apply their diagram-led teaching style to AI agents: systems in which language models plan, call tools, remember context and act with some independence. The book's first part dissects a single agent. It covers how LLMs work internally, how reasoning models use extra computation at inference time, how memory and retrieval are engineered, how agents call tools through function calling and the Model Context Protocol, how planning and reflection loops such as ReAct operate, and how to evaluate agents with benchmarks, LLM judges and trajectory analysis.
The second part turns to specialized designs: multi-agent collaboration, multimodal understanding, and coding agents of the kind measured by SWE-bench. Throughout, readers build a small agent called TinyAgent from scratch using only LLM calls, adding memory, tools and autonomy one component at a time in companion notebooks that run on a free Google Colab GPU. The 300-plus custom figures make internal mechanisms visible without heavy mathematics. Engineers who already use agent frameworks gain a clear picture of what those frameworks do underneath, and technical leads gain a shared vocabulary for design reviews. Because the field moves quickly, specific models and protocols will date faster than the underlying concepts.
Scorecard
Editorial score
How it scored
Editorial score: 75.8 / 1005 weighted criteria
- 18.0/30
Influence30%
- 20.0/25
Authority25%
- 15.0/20
Practical value20%
- 14.3/15
Currency15%
- 8.5/10
Readability10%
- 75.8/100
Editorial score
How the rank was computed
Rank No. 334 of 500 comes from a composite of 51.9: observable signals 17.8 of 55 points plus editorial review 34.1 of 45 points.
All 11 inputs
| Input | Points |
|---|---|
| Observable signals | 17.8/55 |
| Reach (Amazon ratings volume) | 4.2/16.5 |
| Consensus across best-of lists | 2.8/13.8 |
| Awards and bestseller evidence | 0.0/8.3 |
| Reader rating, sample-adjusted | 5.7/8.3 |
| Amazon sales rank | 2.3/5.5 |
| Recency | 2.8/2.8 |
| Editorial review | 34.1/45 |
| Author authority | 9.0/11.3 |
| Influence | 8.1/13.5 |
| Practical value | 6.8/9.0 |
| Readability | 3.8/4.5 |
| Currency | 6.4/6.8 |
Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.
Verdict
A heavily illustrated, code-backed tour of the full AI agent stack, best for engineers who want to understand what frameworks hide.
Strengths
- More than 300 custom explanatory illustrations
- Builds one agent incrementally across chapters
- Free Colab notebooks for every chapter
Keep in mind
Readers seeking production deployment guidance or tutorials for a specific agent framework will need other resources, and model-specific details will age quickly.
Key takeaways
- An agent is an LLM wrapped with memory, tools and planning; understanding each component separately makes agent behavior and failures easier to diagnose.
- Reasoning models improve results by spending more computation at inference time, which changes cost and latency trade-offs for agent design.
- Memory is an engineering choice: agentic retrieval and context engineering decide what an agent knows at each step.
- Evaluating agents requires examining full action trajectories and repeated-run reliability, not just final answers, because the path an agent takes matters as much as its result.
Notable facts
- The book carries endorsements from Chip Huyen, author of AI Engineering, MIT EECS assistant professor Omar Khattab, and Meta FAIR research scientist Ofir Press, coauthor of SWE-bench. Source (opens in a new tab)
- The official code repository for the authors' previous O'Reilly book, Hands-On Large Language Models, has more than 30,000 GitHub stars. Source (opens in a new tab)
About the author
Questions
What is An Illustrated Guide to AI Agents about?
Do I need to read Hands-On Large Language Models first?
Who should read An Illustrated Guide to AI Agents?
Related books
- HBR Guide to Generative AI for Managers Elisa Farri and Gabriele Rosani
- Designing Multi-Agent Systems Victor Dibia
- Building LLMs for Production Louis-François Bouchard and Louie Peters
- Prompt Engineering for Generative AI James Phoenix and Mike Taylor
- Generative AI on AWS Chris Fregly, Antje Barth and Shelbee Eigenbrode
- Generative AI System Design Interview Ali Aminian and Hao Sheng
Sources
- An-Illustrated-Guide-To-AI-Agents - official code repository (GitHub) (opens in a new tab)
- An Illustrated Guide to AI Agents - Helion (O'Reilly listing and table of contents) (opens in a new tab)
- O'Reilly Illustrated Guide to AI Agents - BERI learning resource page (opens in a new tab)
- HandsOnLLM organization - GitHub (opens in a new tab)
As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.






