Asked nicely
not JSON“Order 1042 status as JSON:”
Lober release: the Portal. Q:
No schema: prose, because that’s what it learned.
Module 10 · Structured outputs
Make a model answer in valid JSON, every time
Asking a model for JSON gets you JSON most of the time. A schema mask gets it every time: at each step it blocks the tokens the schema doesn’t allow. Here you watch a tiny model write under one, token by token, and see what it guarantees and what it doesn’t.
Read as
The whole idea in one example
Same model, with and without a schema
Under a schema
valid“Where is order 1042?”
{"status":"delayed"}A mask at every step: only the schema’s tokens get through.
Larkspur mini is tiny and has never seen JSON, so its prose is odd and its picks are guesses. That is the point: the mask makes even this model’s answers valid. Whether they are right is another question, which chapter 6 asks. Every answer here is computed live in your browser from Larkspur mini; none is any provider’s.
How it works
Four ideas carry the module
1 Asking isn’t enforcing
With no schema, the model writes whatever it finds likely
OpenAI: JSON mode gives valid JSON, but only Structured Outputs ensure the answer follows your schema.
Source, checked 21 October 2026
2 The mask
At each step the schema lets some tokens through and stops the rest
Anthropic: structured outputs guarantee schema-compliant responses through constrained decoding; your JSON schema is compiled into a grammar that constrains Claude's output.
Source, checked 21 October 2026
3 Tokens, not characters
The mask checks every byte of a token, so long ones stop at a length cap
XGrammar (Dong and colleagues, 2024) runs context-free grammars for constrained decoding; it splits the vocabulary into tokens that can be checked ahead of time and tokens checked at run time, for up to 100× speedup over earlier engines.
Source, checked 21 October 2026
4 Valid isn’t right
The mask forces a value; it can’t make it true
Anthropic: when the answer hits max_tokens, the output may be incomplete and not match your schema; retry with a higher max_tokens.
Source, checked 21 October 2026
The story
Larkspur’s order system wants JSON
Larkspur Supply Co.’s order system wants the assistant’s answers as JSON, so the team makes Larkspur mini, a tiny model running in your browser, answer in a schema. You’re on the team that has to make every answer readable by a program, and find out what that does and doesn’t promise.
Played Tool calling? The schemas here are the same subset its host checks arguments with. Tokens showed why a token is not a character; chapter 4 here is where that bites.
- 768
- tokens the mask judges at every step
- 32
- tokens in Larkspur mini’s window: prompt and answer
- 19
- tokens in its masked answer to “Where is order 1042?”
- 50 µs
- of CPU per token for a real engine’s mask over 128k tokens (llguidance)
Your path
The chapters
Chapter 2 · done
A schema is a contract
What does checking the answer after it is written buy you, and what does it cost?
Chapter 4 · done
Tokens are not characters
How does a mask written for characters judge a token that carries several?
Chapter 5 · done
Choices from a list
What changes when a field is a choice from a list instead of free text?
Chapter 7 · done
Workbench: Larkspur’s order schema
Which schema gets a valid and right answer for every order?
Who it’s for
- Engineers
- Product managers
- Analysts
- No code needed
Engineers, product managers and analysts whose programs read a model’s answers. It builds on Tool calling (schemas) and Tokens (what a token is); any maths is optional.