TinkerLoading Larkspur mini
Each layer’s MLP has its own fuse.
Chapter 6 of 7
One neuron, many meanings
Heads and blocks are big parts. The team looks smaller: each MLP has 128 neurons. Surely one of them means “shipping”?
This chapter: Does one neuron stand for one idea?
About this chapter
You will learn
- What a neuron is, and how its top tokens are found
- Why most neurons mix unrelated tokens
- Why researchers look for features instead
Words you’ll meet
- Neuron
- One of the numbers in an MLP’s middle layer; it fires (goes high) on some tokens more than others.
- Polysemantic
- Elhage and colleagues (2022) call it polysemanticity when one neuron responds to several unrelated things, and explain it with superposition: more features than neurons.
- Feature
- A direction in the model’s activations that stands for one idea, found by methods that go beyond single neurons.
Step 1 of 5
Each block ends in an MLP of 128 . A neuron fires more on some tokens than others.
Open a neuron.
Read over every window of Larkspur’s training documents. Larkspur mini’s numbers, computed live in your browser: a tiny model, not any provider’s.