TinkerLoading Larkspur mini
Layer 1 · head 1 · no tokens yetType a prompt and trace it.
Chapter 7 of 7
Trace a prompt
You have seen every part of Larkspur mini. Now the whole machine is yours: type any prompt and follow it through, from tokens to the next token.
This chapter: Can you follow any prompt through the model, step by step?
You will learn
- How the pieces fit: tokens, vectors, attention, layers, the next token
- What a prompt longer than the window loses
- How to read what a head is doing on text you chose
Words you’ll meet (2)
- Trace
- Everything one run of the model computed for a prompt: its tokens, every attention pattern and the next-token probabilities.
- Truncation
- Dropping the oldest tokens of a prompt that doesn’t fit the window. The model never reads them.
The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.
Step 1 of 5
The whole of Larkspur mini is yours. Type any prompt (or keep the one in the box) and trace it. The box counts tokens live: the model reads at most 32.
12 of 32 tokens
- Q
- u
- estions
- ␣g
- o
- ␣to
- ␣the
- ␣b
- ill
- ing
- ␣team
- ␣through
Loading Larkspur mini…