What Is ChatGPT Doing ... and Why Does It Work?
Synopsis
Stephen Wolfram expanded this short book from an essay he published in February 2023, less than three months after ChatGPT's release made large language models a public topic. He starts from the core mechanism: the model repeatedly estimates the probability of each possible next token and picks one, with a temperature setting that adds controlled randomness. He then explains why such probabilities cannot simply be counted from text and must come from a model, and walks through neural networks, training by gradient descent, embeddings that place words in a geometric meaning space, and the transformer architecture with its attention mechanism. Many illustrations use the smaller GPT-2 model.
The second half asks why such a simple procedure produces fluent language. Wolfram suggests that human language has more regular structure than commonly assumed, a semantic grammar that the network captures implicitly, and he connects this to his idea of computational irreducibility, arguing that some computations cannot be shortcut and that neural networks therefore have limits. A final section proposes pairing ChatGPT with Wolfram|Alpha for exact computation and curated facts. Readers gain a concrete mental model of next-token generation in roughly a hundred pages, along with a sense of the open scientific questions it raises.
Scorecard
Editorial score
How it scored
Editorial score: 73.0 / 1005 weighted criteria
- 24.0/30
Influence30%
- 20.0/25
Authority25%
- 13.0/20
Practical value20%
- 9.0/15
Currency15%
- 7.0/10
Readability10%
- 73.0/100
Editorial score
How the rank was computed
Rank No. 115 of 500 comes from a composite of 62.0: observable signals 29.0 of 55 points plus editorial review 33.0 of 45 points.
All 11 inputs
| Input | Points |
|---|---|
| Observable signals | 29.0/55 |
| Reach (Amazon ratings volume) | 11.5/16.5 |
| Consensus across best-of lists | 5.5/13.8 |
| Awards and bestseller evidence | 5.0/8.3 |
| Reader rating, sample-adjusted | 4.1/8.3 |
| Amazon sales rank | 1.2/5.5 |
| Recency | 1.7/2.8 |
| Editorial review | 33.0/45 |
| Author authority | 9.0/11.3 |
| Influence | 10.8/13.5 |
| Practical value | 5.9/9.0 |
| Readability | 3.2/4.5 |
| Currency | 4.1/6.8 |
Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.
Verdict
A compact, clear account of how language models generate text from a leading computational scientist, though its closing section promotes Wolfram|Alpha.
Strengths
- Short, clear explanation of next-token prediction
- Concrete visual examples built on real models
- Connects mechanics to deeper questions about language
Keep in mind
Examples rely on GPT-2 and early ChatGPT, the closing Wolfram|Alpha section reads as promotional, and newcomers may find the neural-network passages dense.
Key takeaways
- A language model writes by repeatedly choosing a plausible next token from a probability distribution, building text one token at a time.
- A temperature setting that sometimes picks less likely tokens makes output more varied and natural than always choosing the top prediction.
- Embeddings represent words and passages as points in a high-dimensional space where related meanings sit close together, which helps the network generalize.
- Computational irreducibility implies that neural networks cannot shortcut every computation, so exact tools such as Wolfram|Alpha can complement them.
Notable facts
- The book expands an essay of the same title that Wolfram published on February 14, 2023, on his Stephen Wolfram Writings site. Source (opens in a new tab)
About the author
Questions
What does Wolfram's What Is ChatGPT Doing explain?
Do I need a technical background to read Wolfram's ChatGPT book?
Is What Is ChatGPT Doing still accurate in 2026?
Related books
- Hands-On Large Language Models Jay Alammar and Maarten Grootendorst
- Build a Reasoning Model (From Scratch) Sebastian Raschka
- The Scaling Era Dwarkesh Patel and Gavin Leech
- Vibe Coding Gene Kim and Steve Yegge
- LLM Engineer's Handbook Paul Iusztin and Maxime Labonne
- The Hundred-Page Language Models Book Andriy Burkov
Sources
As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.






