TinkerLoading Larkspur mini
Please contact the … · not steered
Chapter 5 of 7
Steering
If information sits in the residual stream, can the team push it there? They build a direction from two sets of Larkspur prompts.
This chapter: Can you change what a model says without changing its weights?
About this chapter
You will learn
- How a steering direction is made from two sets of prompts
- What adding it does, and what too much does
Words you’ll meet
- Steering
- Adding a vector to the model’s residual stream during a run to push its output one way.
- Direction
- A vector found as the average of one set of runs minus the average of another.
Step 1 of 5
pushes a model without retraining it. The : shipping prompts’ average activation minus billing prompts’.
Build the direction.
Loading Larkspur mini