Interactive course14 lessons · no prerequisites
See what a
language model
actually does.
Follow one sentence through a language model, from raw text to the next token it bets on. Then zoom out: context, tools, and one simple loop turn that model into the chatbot you talk to.
Sentenceeat an
- 01
Text
eat an - 02
Tokens
eatan - 03
Vector
an · [0.50, 0.00] - 04
Next bet
orange66.7%
One course trace. Four stages you will make tangible.
Our method
Try it first. Understand it next.
You predict, manipulate, then name what you have just seen. Theory arrives once intuition has taken shape.
- 01
Manipulate before naming
Start with a concrete example, risk a prediction, then manipulate the mechanism. The explanation makes sense of what you just saw.
- 02
Build without black boxes
You never meet a mechanism you have not already built by hand. By chapter 14, you retrace the inner machine and its outer system on a brand-new phrase.
- 03
Watch the computation take shape
Tokens, vectors, weights, and probabilities stay visible while you manipulate them.
Course map / 14
The learning path
A progressive path from raw text to the assistant. No AI background required. You don’t need to be a mathematician.
From text to representations
The game a language model plays
No neurons yet: count what follows each word and run the generation loop yourself.
Tokenization, or how a model reads
Characters, words, or subwords: three ways to break a sentence, three different bills.
Embeddings: meaning becomes geometry
Drag words around a toy map and watch resemblance become an angle.
Inside the Transformer
Attention, where tokens start talking
Same word, two sentences: watch attention build two different vectors.
Word order enters the machine
Reverse the same tokens and prove exactly why a Transformer needs a position signal.
The Transformer block, the floor a model stacks
One floor of the tower, walked end to end - and the quiet trick (amend, never replace) that lets a hundred floors stack.
Prediction and learning
Logits, temperature, and the token that comes out
The final vector becomes a bet over the whole vocabulary - and the untrained toy has a surprise in store.
Training, the machine that fixes its own numbers
Loss, gradient, small step, repeat: watch chapter 7’s absurd bet flip.
What scale buys
The same recipe, a thousand times bigger: see where the parameters live and what the data, compute, and a training run actually cost.
From raw continuer to assistant
A model that only predicts the next token, coaxed into a helpful assistant: example conversations first, then human preferences - and the limits of each.
From model to chatbot
The prompt is the program
Change instructions and examples in context without touching the frozen model weights.
Reasoning - buying time to think
Watch intermediate tokens carry a result into later bets, then measure the price and limits of that scratchpad.
Confident, wrong, and augmented
Make the toy hallucinate, add retrieved evidence, and see why grounding helps without guaranteeing truth.
Trace a sentence yourself
Follow a brand-new phrase end to end, then separate the model, its context, and the chatbot orchestrator.