Tinker Post-training

by Samy (Kondasamy Jayaraman) · drawn with Colorfigures by April Zhu

...

The exam hall: the model, a stack of cards per written answer as tall as its odds, and a grader’s desk.

Chapter 1 of 7

Base or assistant

Larkspur wants an assistant of its own. Pre-training gave it a base model that has read a mountain of text, but it doesn’t answer customers yet.

This chapter: What is the difference between a base model and an assistant?

About this chapter

You will learn

  • Why a base model continues text instead of answering
  • What supervised fine-tuning on examples changes
  • What post-training adds after that

Words you’ll meet

Base model
A model trained only to guess the next words of documents. It continues text; it was never taught to answer.
Post-training
Everything done to a base model after pre-training to turn it into a helpful assistant.
Supervised fine-tuning
Training on examples people wrote: a prompt and the reply they want. The model learns to copy their style.

Step 1 of 4

A customer asks about refunds. A continues text: it learnt only to guess the next words.

Press Ask the base model.

The customer asks

“How long do standard customers have to ask for a refund?”

The toy policy is a choice among answers written for this site, not a language model. Its updates run live in your browser.