The window
1,074 tokensChapter 1’s turn, counted by this site’s tokenizer.
Module 8 · Context
What the model reads each turn, and what it remembers
Every turn, the host builds the model a window: the system prompt, the tools, the conversation so far, the documents and the question. It has one budget. Here you choose what goes in, in what order, and what is kept outside for later.
Read as
The whole idea in one example
Everything the model reads shares one window
The answer
right“Above what order total is parcel shipping free?”
Parcel shipping is free on orders over $500. Freight is quoted per order by weight and distance.
The recorded answer to that window.
Anthropic: everything in the request counts toward the context window: the system prompt, every message (tool results and documents included), the tool definitions, and the reply the model writes. The counts here come from this site’s own tokenizer, so they show the mechanism, not any provider’s numbers. Source, checked 21 October 2026.
How it works
Six ideas carry the module
1 One budget
System, tools, history, documents and the reply share it
Anthropic: everything in the request counts toward the context window: the system prompt, every message (tool results and documents included), the tool definitions, and the reply the model writes.
Source, checked 21 October 2026
2 More isn’t better
Distractors around the fact make answers worse
Anthropic: more context isn't automatically better; as the token count grows, accuracy and recall degrade, which it calls context rot.
Source, checked 21 October 2026
3 Fetch when needed
Search for the passage instead of sending every document
Anthropic's context-engineering guide: agents can keep lightweight references (file paths, stored queries) and load the data just in time, when they need it.
Source, checked 21 October 2026
4 Stable first
A prefix that repeats is read from the cache
OpenAI: put stable instructions and shared reference material first and dynamic content, like timestamps, at the end, because a cache hit needs the whole prefix to match.
Source, checked 21 October 2026
5 Clear the desk
Old tool results go; the decisions stay
Anthropic's context editing can clear old tool results once the context passes a threshold (100,000 input tokens by default), keeping the most recent ones (3 by default) and leaving a placeholder where each was.
Source, checked 21 October 2026
6 Memory outside
Notes in files, read in the next conversation
Anthropic's memory tool lets Claude create, read, update and delete files under /memories that persist between conversations.
Source, checked 21 October 2026
The story
Larkspur’s window fills up
Larkspur Supply Co.’s assistant now answers from documents, calls tools and holds long conversations, so the team has to choose what goes into its window each turn. You’re on the team that sets its context policy: what to retrieve, how to keep the history, where to cache and what to remember.
RAG chapter 2 showed why you can’t send everything; Agents chapter 5 summarised a long history. This module builds on both.
- 8,000
- tokens in Larkspur’s window budget, the reply’s room included
- 0.1×
- the input price for tokens read from the cache
- 1.25×
- the input price to write them to the 5-minute cache
- 4
- cache breakpoints a request can set
Your path
The chapters
Chapter 1 · done
What’s in the window
What does the model actually read on each turn, and how much room does each part take?
Chapter 3 · done
Fetch it when you need it
Should the assistant get every document up front, or look things up when it needs them?
Chapter 4 · done
Order for the cache
How can Larkspur stop paying full price for the same text on every call?
Chapter 5 · done
Clearing the desk
What can the assistant drop from a long conversation without losing what matters?
Chapter 6 · done
Memory outside the window
How can the assistant remember something from one conversation in the next?
Chapter 7 · done
Workbench: the context policy
Which context policy gives Larkspur good answers for the fewest tokens and the lowest cost?
Who it’s for
- Engineers
- Product managers
- Analysts
- No code needed
Product managers, engineers and analysts whose assistant already answers from documents and calls tools. RAG and Tool calling come first; no code or maths needed.