Small adapters

The fine-tune worked: Larkspur mini signs off. But it changed every weight in the model, and the team would need a whole copy of the model for every style they train.

This chapter: How can training change a model’s replies while touching only a small share of its weights?

You will learn

How many weights a full fine-tune trains, counted from the model’s shapes
What LoRA trains instead, and why that is so much smaller
Why an adapter can be clipped on and off one shared base model

Words you’ll meet (4)

Full fine-tune: Training that changes every weight in the model, so the result is a whole new copy.
LoRA: LoRA freezes a pre-trained model's weights and adds small trainable low-rank matrices to each layer of the Transformer.
Adapter: The small set of weights LoRA trains. It sits beside the base model, which stays as it was.
Rank: How thin LoRA’s matrices are. Rank 4 means each adapted matrix gets two thin matrices 4 wide.

The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.

Step 1 of 5

Chapter 2’s fine-tune changed every weight in Larkspur mini. That is a full fine-tune: the result is a whole new copy of the model. Count what it trained.