TinkerThe 6 multi-step questions, ready to sweep.
Thinking recorded
Run a question to see what the model wrote before it answered.
Chapter 3 of 7
The effort dial
Thinking costs tokens and time. Larkspur’s team can ask for more or less of it with one setting, effort, and wants to know what each step of the dial buys on the questions that need it.
This chapter: Does more thinking keep making answers better?
You will learn
- What an effort setting is: none, low, medium or high
- How to sweep a set of questions across every effort level
- Why accuracy rises and then flattens while the cost keeps climbing
Words you’ll meet (3)
- Effort
- A setting that tells a reasoning model how much work to put into an answer, thinking included.
- Sweep
- Running the same questions at every setting, to compare them on equal terms.
- Accuracy
- The share of answers that pass the check: here, the Evals module’s key-fact check.
The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.
Step 1 of 3
Effort is the setting that tells a reasoning model how much work to put into an answer, thinking included: here none, low, medium or high. Larkspur’s test set for it is 6 multi-step questions: quotes with volume prices, stock across warehouses, delivery dates: several steps, each easy to slip on. Before the sweep: