TinkerLoading Larkspur mini
We reply within five business … · “days” marked in the Lens tab
Chapter 3 of 7
The guess forming
Fuses showed which parts matter. Now the team wants to see the guess itself take shape, layer by layer, through Larkspur mini’s 2 blocks.
This chapter: Where inside the model does the answer first appear?
About this chapter
You will learn
- How the logit lens reads a guess partway up
- What the model guesses before any block has run
- Which block makes the guess in Larkspur mini
Words you’ll meet
- Block
- One layer of the model: its attention heads, then its MLP, each adding to the residual stream.
- Logit lens
- nostalgebraist's logit lens (2020) reads a model's middle layers through its own final output step, and finds a fair guess often forms partway up and is refined after.
Step 1 of 5
The reads the guess partway up, as if the model stopped there and answered.
Open the Lens tab.
Loading Larkspur mini