Module 11 · Reasoning
Reasoning models write out their working first. Here you watch that thinking fix a quote, count what it costs in tokens and time, see it go wrong on simple questions, and pick an effort for each kind of request with an eval.
The whole idea in one example
“A customer orders 55 Kestrel K2 scanners as two order lines, one of 30 and one of 25. How much more do they pay than if all 55 were on one line?”
Nothing more: 55 scanners get the 50+ price either way.
No thinking. Answers at once and misses that volume prices apply per order line.
“A customer orders 55 Kestrel K2 scanners as two order lines, one of 30 and one of 25. How much more do they pay than if all 55 were on one line?”
They pay $1,925 more: two lines of 30 and 25 each take the $445 price ($24,475), while one line of 55 takes $410 ($22,550).
150 thinking tokens first. Applies the per-line rule and gets the difference.
What the assistant read: “Price list, 2025 (1-9 units / 10-49 units / 50+ units): Kestrel K2 handheld scanner $480 / $445 / $410. Volume prices apply per order line.” Both answers are recorded and graded by the Evals module’s key-fact check. Token counts come from this site’s tokenizer, and the waits from a speed Larkspur assumes; they teach the mechanism, they are not any provider’s numbers.
How it works
A reasoning model writes out its working before the answer
With adaptive thinking, Claude weighs each request and decides whether to think and how much: a simple factual question may get no thinking at all, and a multistep problem gets deeper reasoning.
Source, checked 21 October 2026
Billed as output, and written before the first word of the answer
Anthropic bills the tokens Claude uses while thinking as output tokens, and bills the full thinking even when the response shows only a summary of it.
Source, checked 21 October 2026
More effort helps, then flattens, while the cost keeps climbing
Snell and colleagues (2024) found that how much extra thinking-time compute helps depends on how hard the question is, and that spending it well per question made it more than 4 times as efficient as a best-of-N baseline.
Source, checked 21 October 2026
On simple questions with distractions, more thinking can hurt
Gema and colleagues (2025) built tasks where longer reasoning makes large reasoning models less accurate, among them simple counting tasks with distractors: an inverse scaling between thinking-time compute and accuracy.
Source, checked 21 October 2026
The model decides how much each question needs
Anthropic’s docs call effort a behavioural signal, not a strict token budget: at lower effort Claude still thinks on hard problems, but less than it would at higher effort.
Source, checked 21 October 2026
The lowest effort that passes the gate, kind by kind
Anthropic’s docs advise testing each effort level on your own use cases before deploying, since its impact varies by task type.
Source, checked 21 October 2026
The answers and their thinking were written for this site, per question and effort, to illustrate the cited findings; they were not measured on a real model. The grading, token counts, costs and waits run live in your browser.
The story
Back at Larkspur Supply Co., the assistant starts getting quotes and delivery questions with several steps, and the team has to decide how much it should think before it answers. You’re on the team that measures what thinking buys and sets the dial for each kind of request.
Evals built the test set and the release gate this module gates effort with.
Your path
Chapter 1 · done
What changes when a model writes out its working before it answers?
Chapter 5 · done
What does an effort setting actually control, in today’s APIs?
Chapter 6 · done
How do you choose an effort for each kind of question, with numbers?
Product managers, engineers and analysts choosing how much a model should think, and paying for it. Evals comes first; no code or maths needed.