In this lesson
Reasoning - buying time to think
Intermediate tokens can carry earlier work into later bets. Test when that helps, what it costs, and why a fluent trace can still lie.
By the end, you can
- Explain how an autoregressive scratchpad carries intermediate state into later generation steps.
- Explain how longer traces and multiple samples spend test-time compute without updating the model's weights.
- Separate prompted reasoning from trained reasoning without treating one training recipe or a visible trace as universal.
Before you start: The prompt is the program
A model cannot rewrite itself while it answers. It can still improve its odds on a hard question, and the loop you ran in chapter 7 is why. The weights froze long before you typed; the working state keeps moving.
Each generated token re-enters as context, so a model can write an intermediate result and bet on top of it. Call every extra evaluation a coin of computation: one coin buys one more pass, at a price that climbs with the context.
Some products print those intermediate steps, others summarize or bury them. What you buy underneath is the same either way: work at answer time, spent to move the odds.
Observe a two-hop question
The course trace sits this one out - “eat an orange” never needed two linked facts. This lab asks where Ana lives. First find the Blue Tower’s city; then connect Ana to that city.
The logits in this lab are authored rather than measured from a production model. The instrument lets you compare what its two answer modes do with the same stated facts.
Predict what thinking first will do
Your bet
In this authored two-hop fixture, which mode gives the correct city a better shot?
Commit before manipulating the instrument. This choice is final.
Each emitted token starts another autoregressive step and becomes context for later bets. Choose a question, then answer directly or keep one sampled trace as its judge.
Live missions
- Turn on thinking and win the two-hop bet
- Stretch: make thinking backfire on the one-hop question
- The Blue Tower stands in Mira.
- Ana lives in the Blue Tower’s city.
user: Where does Ana live?
Answering directly: this fixture emits no intermediate scratchpad, so its authored direct logits cover the facts at once.
Answer distribution
In this fixture, the authored direct logits hedge and the wrong city edges ahead. Try thinking first.
Authored fixture: a kept trace changes context, then final logits. It measures no production model and does not claim every two-hop task needs a scratchpad. Real systems may hide traces and use votes or verifiers.
Manipulate the thinking budget
Start on the two-hop question in answer directly. In this fixture, the wrong city edges ahead. The bars run a real softmax; the logits underneath were written by hand, no production model consulted.
Switch to think first. The lab lays out its authored traces, then hands you the gavel. Keep the one that writes the missing city into the context and watch the final bet swing to the correct answer.
Raise thinking samples and watch the token counter climb. This is best-of-N: draw N candidate traces and keep one.
At the default two samples, one trace resolves the hop and one is fluent but wrong; five samples spend five traces. In production, batching, caching, and context length reshape the bill.
Keep the fluent wrong trace instead. Its claim also enters the context, and the wrong answer wins. More candidates help only when a judge, a vote, or a verifier selects the useful work.
Now choose the one-hop control. Its direct bet is already right. A useful trace leaves it unchanged; a wrong one can flip it. On this question, the only way to lose is to think about it.
Primary. On the two-hop question, turn on thinking and keep a trace that makes the correct answer win.
Stretch. With at least two samples on the one-hop control, keep a fluent wrong trace and watch it flip an answer that was already right.
Compare your answer
The direct two-hop logits were authored to hedge. A useful trace writes the missing city into context, so the next bet can condition on it.
More samples put more candidates on the table; someone still has to keep useful work. On the one-hop fixture, extra text adds no state and a wrong trace can steer the bet away.
You have demonstrated the lab’s arithmetic, nothing more. It does not prove that every two-hop question needs a scratchpad or that one-hop reasoning is harmful in general.
Explain where the extra computation comes from
Each generated token requires another autoregressive evaluation. With a KV cache, systems usually reuse earlier keys and values, but the new token still attends over a growing cache. A coin bought late costs more than a coin bought early.
Intermediate tokens can act as an external scratchpad. A later step receives their result as ordinary context instead of reconstructing it from nothing. The layers still do the heavy lifting; the scratchpad spares them the restart.
Chain-of-thought names the intermediate reasoning steps themselves, whether a prompt elicits them or training shapes them.
Thinking tokens are the coins a system spends before the final answer. Depending on the product, raw reasoning may be shown, summarized, or hidden. Hidden does not mean free; visible does not mean faithful.
Test-time compute is broader still. A system may use a longer trace, sample several candidates, search a tree, run a verifier, or call a tool. These choices spend work at answer time without updating the model’s weights.
In best-of-N, whoever writes the scorer decides what counts as best.
DeepSeek-R1 is a published example of reinforcement learning on verifiable rewards, RLVR for short. Math answers and code tests provide signals that can train useful reasoning behavior.
Other reasoning models may combine supervised examples, distillation, reinforcement learning, search, or different post-training pipelines. RLVR names one path to the behavior.
A prompted base model can produce a useful scratchpad too. Training for reasoning shapes that behavior into the weights, though the recipe and the exposed controls vary from system to system.
A fluent chain can land on a false result. Causal tests of chain-of-thought show it can also misreport what actually produced the answer.
Read the visible steps as output you can inspect: they can help, and they can expose an error. What they cannot do is testify to the cause of the answer.
Reflect on bought computation
Checkpoint
Why can intermediate reasoning tokens help with a hard question?
Pick an answer first.
In the lab, you elicited the trace and kept the winner. Training can build that habit into the weights instead. Which recipe puts it there is another question.
Checkpoint
Which statement correctly describes reasoning models?
Pick an answer first.
Connect thinking to the outside world
Reasoning can spend more work on what the weights and current context contain. It cannot conjure a fact that neither one holds. The next lesson adds retrieval and tools, then asks who fetches the evidence and what can still go wrong.
Sources and scope
- Wei et al. (2022) reports chain-of-thought prompting gains on selected multi-step benchmarks for sufficiently large language models.
- Wang et al. (2022) samples diverse reasoning paths and selects the most consistent final answer.
- Kwon et al. (2023) describes KV-cache management in production LLM serving, where decode cost depends on more than a fixed token count.
- DeepSeek-AI (2025) documents reinforcement learning with verifiable rewards as one reasoning-model training recipe.
- Lanham et al. (2023) uses causal tests to show that chain-of-thought can be an unfaithful explanation of a model’s answer.
- The Blue Tower, cities, traces, logits, and percentages are hand-authored. The lab shows only that its kept trace changes the context and therefore its authored final bet.
- Content and claims reviewed on July 28, 2026.