TinkerFour drafts for an upset customer, each as tall as the toy policy’s odds of writing it.
Chapter 2 of 7
Which is better?
Examples teach a style, but writing a perfect reply for every case is slow. Comparing two drafts is fast: anyone can say which one they prefer.
This chapter: How do people’s preferences become something a model can learn from?
About this chapter
You will learn
- What a preference pair is
- How a reward model turns pairs into scores
- Why pairs suit tone, where there is no right answer
Words you’ll meet
- Preference pair
- Two drafts for the same message, with a person’s pick of the better one.
- Reward model
- A scorer fitted to preference pairs: drafts that win their pairs score higher.
Step 1 of 4
Every draft has the fact; only the tone differs. A asks which of two is better.
Pick the better draft in each pair.
The customer asks
“My order 1042 is late AGAIN. Third time this year!”
The toy policy is a choice among answers written for this site, not a language model. Its updates run live in your browser.