Tinker Small models: distillation & routing

by Kondasamy Jayaraman · drawn with Colorfigures by April Zhu

...

Two tiers of Larkspur’s assistant: large at the back, small in front.

Chapter 4 of 7

Send it the easy ones

Larkspur mini and its student showed the mechanism. Now the real question: Larkspur’s assistant comes in two tiers, a small model and a large one, both reading the same documents. Which questions can the small tier take?

This chapter: Where does a small model answer as well as a large one, and where does it fall short?

You will learn

  • How two tiers of the same assistant compare on Evals’ test set
  • What a small tier costs against a large one, from a real price list
  • Which kinds of question a small model gets wrong
Words you’ll meet (2)
Tier
One size of model offered for the same job: here, a small tier and a large tier.
Relative cost
Cost counted in large answers: one large answer is 1, a small one a fraction of that.

The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.

Step 1 of 3

Larkspur’s assistant comes in two tiers: a small model and a large one, both reading the same retrieved documents. A small answer costs 0.2 of a large one, from a real price list below; from here on costs are in large answers (1 each). The tiers are illustrations written for this site, not measured models. Before you run them:

Anthropic lists Claude Haiku 4.5 at $1 per million input tokens and $5 per million output tokens, and Claude Opus 4.5 at $5 and $25: the smaller model costs a fifth as much per token, in and out. Source: platform.claude.com (checked 2026-10-21)

Run Evals’ 12 questions through both tiers. Both read the same documents. How will the small tier do against the large one?