Kept on the small tier recorded answer
right“Can enterprise customers get a refund?”
No. The refund policy does not apply to enterprise contracts; those customers go to their account manager.
Small tier’s confidence 0.92, cost 0.2
Module 12 · Small models
Do you need the big model for every question?
A small model can learn from a big one, and answer the easy questions for a fraction of the price. Here you compare a tiny model with its student, then route Larkspur’s questions between a small tier and a large one, and prove the cheaper setup loses nothing.
Read as
The whole idea in one example
Two questions, one router
Sent up to the large tier recorded answer
right“What is the hotel nightly limit for business travel?”
Under the 2025 travel policy, lodging is reimbursed up to $210 per night before taxes in most cities and up to $290 in high-cost cities. The older $150 limit is from the 2018 policy.
Small tier’s confidence 0.84, cost 1.2
The router sends a question up when the small tier’s confidence is below 0.9. Both answers are recorded and graded by the Evals module’s key-fact check; the confidence scores are stand-ins set by hand to show the trade-off. Costs are in large answers: a small answer costs 0.2 of one, from the price pair below. The tiers illustrate what the cascade and router papers report; they were not measured.
How it works
Six ideas carry the module
1 Big and small
Fewer weights: less work per token, less sure odds
Anthropic lists Claude Haiku 4.5 at $1 per million input tokens and $5 per million output tokens, and Claude Opus 4.5 at $5 and $25: the smaller model costs a fifth as much per token, in and out.
Source, checked 21 October 2026
2 A teacher’s odds
The student copies how the big model spreads its bets
Hinton, Vinyals and Dean train a small model on the class probabilities a large model produces, used as “soft targets”, raising the temperature of the large model’s final softmax until those targets are suitably soft.
Source, checked 21 October 2026
3 More per example
The odds show which wrong answers are nearly right
The same paper notes that when the soft targets have high entropy, they give much more information per training case than the single right answer does.
Source, checked 21 October 2026
4 Send the easy ones small
A small tier answers the lookups for a fraction
RouteLLM trains routers that pick between a stronger and a weaker model for each query; its authors report cutting costs by more than half in some cases without losing response quality.
Source, checked 21 October 2026
5 Fall back when unsure
Below a threshold, the question goes up
FrugalGPT is an LLM cascade that learns which combinations of models to use for different queries; its authors report it can match the best single model (GPT-4) with up to 98% cost reduction.
Source, checked 21 October 2026
6 Gate the switch
Ship only what keeps every answer that worked
FrugalGPT is an LLM cascade that learns which combinations of models to use for different queries; its authors report it can match the best single model (GPT-4) with up to 98% cost reduction.
Source, checked 21 October 2026
The first chapters run Larkspur mini and its student in your browser. Their numbers teach the mechanism; they are not any provider’s.
The story
Larkspur’s model bill
Back at Larkspur Supply Co., the model bill has grown, and the team wants a smaller, cheaper model to answer what it can without losing the answers that matter. You’re on the team that decides which questions a small model can take, and proves it with the release gate.
Fine-tuning trained Larkspur mini’s style and adapters; Evals built the test set and the release gate. This module makes the assistant cheaper.
- 51,072
- weights in Larkspur mini, the teacher
- 16,112
- weights in its student
- 52%
- held-out places where the student picks what the teacher picks
- 8.4 vs 12
- cost of the router that ships, against all large (relative units)
Your path
The chapters
Chapter 1 · done
Big model, small model
What does a smaller model give up, and what does it save on every token?
Chapter 2 · done
Learning from a teacher
How does a small model learn from a big one, and what does it learn that the text alone doesn’t teach?
Chapter 3 · done
How close is the student?
How do you measure how close a student is to its teacher, on text neither has seen?
Chapter 4 · done
Send it the easy ones
Where does a small model answer as well as a large one, and where does it fall short?
Chapter 5 · done
Fall back when unsure
How does a confidence threshold trade right answers against cost?
Who it’s for
- Engineers
- Product managers
- Analysts
- No code needed
Product managers, engineers and analysts paying a model bill and wondering what a smaller model could take on. Fine-tuning and Evals come first; no code or maths needed.