Tinker Interpretability

by Kondasamy Jayaraman · drawn with Colorfigures by April Zhu

Loading Larkspur mini

Every enterprise account has a named customer success … · the clean run

Chapter 4 of 7

Patching

Swap one word and Larkspur mini’s answer changes. The team wants to know where, inside the run, that word’s effect travels.

This chapter: How do you find where a piece of information sits and where it moves?

About this chapter

You will learn

  • What activation patching is: copying one activation from another run
  • How a grid of patches shows where information travels

Words you’ll meet

Activation
The numbers a part of the model computes for one token on one run, such as the residual stream at one layer.
Activation patching
Copying one activation from a run on another prompt into this run, to see whether the answer follows it.

Step 1 of 5

Swap “customer” for “Kestrel” and the guess flips from “manager” to “team”. Where does that change travel?

Run both prompts.

Loading Larkspur mini