What’s in the window

Larkspur’s assistant now answers customers from its documents, calls tools and keeps a conversation going. Each of those adds text to what the model reads, and the team has started to wonder how much it all takes.

This chapter: What does the model actually read on each turn, and how much room does each part take?

You will learn

The parts of a turn: system prompt, tools, history, documents and the question
That every part, the reply included, shares one token budget
Which parts are big, counted live

Words you’ll meet (5)

Context window: Everything the model reads in one turn, plus the reply it writes. It has a fixed size, counted in tokens.
Token: The unit a model reads and is billed in: a word, part of a word or a mark. Counts here come from this site’s own tokenizer.
System prompt: Larkspur’s standing instructions to the model, sent at the top of every turn.
Tool definition: A tool’s name, description and input schema, sent with every turn so the model knows it can call it.
Budget: How many tokens the host lets one turn use. A window over it is refused before the model is called.

The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.

Step 1 of 6

Mara Quill asks Larkspur’s assistant: “Above what order total is parcel shipping free?” A model reads one block of text per turn: the context window. Everything it knows about this question has to be in there. Right now the turn holds two parts: Larkspur’s system prompt and Mara Quill’s question. Send it.