TinkerLoading Larkspur mini
Larkspur miniThe prompt: “Where is order 1042?”. The schema: one field, status, on time or delayed.
Chapter 3 of 7
The mask
Retries got the team nowhere. A different idea: don’t check the answer after it is written; stop the model writing anything the schema doesn’t allow, one token at a time.
This chapter: How can a schema stop a model writing invalid JSON, before it writes it?
You will learn
- That the schema allows some tokens at each step and blocks the rest
- How the blocked tokens’ probability goes to the allowed ones
- Why the answer then always fits the schema
Words you’ll meet (2)
- Mask
- At one step of writing, the set of tokens the schema allows. Blocked tokens get zero probability, and the model picks among the rest.
- Constrained decoding
- Writing an answer with a mask applied at every step, so only text the schema accepts can come out.
The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.
Step 1 of 4
Checking after the fact only says no. So stop the bad tokens before they are written. A stencil plate now stands across the press’s belt: the schema. At each step it lets some tokens through and stops the rest. Before you look: