How AI Writes Code
AI code generation is next-token prediction: the model repeatedly samples the most probable next token from a learned distribution over its training data, producing code token-by-token without any internal plan or verification step.
A complete interactive classroom, not just a preview.
Start when you are ready to enter this Stage's 4 scenes and explore, respond, and learn as you go.
How does an AI model actually generate code one token at a time?
You ask an AI assistant 'write me a function to sort a list' — and seconds later, working code appears. Where did that code actually come from?
It looks like the AI 'understands' your request and 'thinks up' code from scratch — but the model was never taught programming rules like a student is. Something else must be happening under the hood.
A token-by-token prediction diagram showing how the model picks each next word of code based on probability, not understanding.
AI generates code by repeatedly asking 'what token most likely comes next?' — pattern completion, not problem-solving.
- training process details
- transformer architecture internals
- fine-tuning and RLHF
- prompt engineering techniques
- 01The Magic IllusionslideSlot 1Hook
Show a screen where typing a coding prompt produces a full, working function in seconds. Ask: how did the AI know to write exactly that code?
- AI produces code from a plain-English prompt
- It feels like understanding
- But the model was never explicitly taught programming rules
PhenomenonAn AI assistant generates a complete, working function seconds after receiving a natural-language request.
QuestionIf the model was never taught coding rules like a student, where does the code come from?
- 02The Tempting MisconceptionslideSlot 2Tension
Surface the intuitive belief: the AI 'reads' the request, 'understands' the problem, then 'writes' the solution — like a human programmer.
- Intuition: AI reasons like a developer
- Implies planning, checking, debugging internally
- But the model has no execution loop and no compiler
PredictionLearners expect AI code generation to involve a planning step where the model figures out the algorithm before writing.
Tempting intuitionThe AI understands the problem and writes code the way a human programmer would — thinking first, then typing.
- 03One Token at a TimeslideSlot 3Reveal
Reveal that the model only does one thing: predict the next token. Show a diagram of a partial code output and a probability distribution over what comes next, with the chosen token highlighted.
- The model outputs one token (a piece of code) at a time
- Each token is chosen from a probability distribution learned during training
- The chosen token is fed back in, and the process repeats
EvidenceEvery code-generating model produces output autoregressively: it sees a sequence and picks the most probable next token, appends it, and repeats.
ConclusionThe 'smart' code is just thousands of tiny next-token guesses stitched together.
Mechanism- 1The prompt and all tokens generated so far are fed into the model, which outputs a probability for every token in its vocabulary.
- 2One token is sampled from that distribution, appended to the sequence, and the loop restarts — no planning step, no verification, just probability sampling repeated thousands of times.
- 04Why It Looks Like MagicslideSlot 4Takeaway
Transfer the insight: when you ask the AI to write a sorting function, it isn't 'figuring out' sorting — it's predicting, from billions of seen code examples, the token sequence that sorts typically looks like. That statistical fluency is why simple requests work and why edge cases sometimes fail.
- Code comes from patterns seen in training data
- Statistical fluency mimics competence
- Unusual requests drift because the pattern is thinner
TransferAsk the AI to write code in a brand-new language or for a brand-new library it likely never saw.
Expected inferenceLearners should expect the output to degrade, drift, or invent plausible-looking but broken APIs — because next-token prediction has no ground truth to fall back on outside its training distribution.
Discussion threads for a Stage aren't available yet.
