Fetch it when you need it

More passages made answers worse, not better. Yet Larkspur can’t know in advance which passage a customer will need.

This chapter: Should the assistant get every document up front, or look things up when it needs them?

You will learn

Stuffing: every document in every window, and what it costs
Just in time: a search tool the model calls when the question needs it
Why a small, focused window answers better

Words you’ll meet (3)

Stuffing: Sending every document up front in every window, whatever the question.
Just in time: Loading a document only when the model asks for it, through a tool.
Search tool: A tool the model calls with a query; the host runs the search and returns the best passages.

The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.

Step 1 of 5

Larkspur doesn’t know which passage a question needs, so the simplest plan is stuffing: send all 15 passages up front, every turn. Mara Quill asks: “How many days after delivery can I still ask for a refund?” Send it with every document.