Big model, small model

Larkspur’s assistant now answers from its documents, passes the release gate and signs off in the team’s voice. Then finance sends the monthly model bill. Someone asks: could a smaller model do the same job for less?

This chapter: What does a smaller model give up, and what does it save on every token?

You will learn

How a model’s size is counted, and why the work per token follows it
What a small model’s top 5 looks like beside a big one’s, on the same sentence
Why “smaller” means “less sure”, not “knows different words”

Words you’ll meet (3)

Weights: The numbers inside a model that decide what it writes. A model’s size is how many it has.
Work per token: The multiplications a model does to write one piece of text. It grows with the number of weights.
Top 5: The five next pieces a model thinks most likely, with the probability it gives each.

The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.

Step 1 of 4

On the bench: Larkspur mini, the tiny model from the Fine-tuning module, and its student, a smaller model taught by it. Both run in your browser. Their numbers teach the mechanism; they are not any provider’s. Weigh them first.