Tinker Calibration

by Samy (Kondasamy Jayaraman) · drawn with Colorfigures by April Zhu

Loading Larkspur mini

Chapter 3 of 7

After the polish

Fine-tuning taught Larkspur mini to sign off in the house style. Its replies look more polished. The team wonders what else moved.

This chapter: Does polishing a model’s style change how far its confidence can be trusted?

About this chapter

You will learn

  • What the style fine-tune did to its confidence
  • Why it is scored on text it was not tuned for
  • What larger models showed after post-training

Words you’ll meet

Fine-tune
Larkspur mini trained a little more on a small set of styled replies.
Overconfident
Says it is surer than it turns out to be: the jars hold less mint than their rings.

Step 1 of 3

The learned only to sign off “Cheers, the Larkspur team”. Score it on held-out text it was not tuned for.

Predict first.

The same held-out text, read by the style fine-tune. How do its jars compare?