Tinker Post-training

by Samy (Kondasamy Jayaraman) · drawn with Colorfigures by April Zhu

...

Every draft is marked mint: each one carries the fact the grader checks.

Chapter 5 of 7

Only what you can check

The grader worked on the warranty. The team points it at the upset customer from chapter 2, hoping it will learn a warmer reply.

This chapter: What happens when a grader can’t tell the good drafts from the bad ones?

About this chapter

You will learn

  • Why equal rewards mean no update
  • Which qualities a grader can check, and which it can’t
  • Why teams mix graders and preferences

Words you’ll meet

Spread
How far apart the rewards in a group are. With no spread, no draft beats the average.

Step 1 of 3

Back to the upset customer. Every draft has the fact, so the key-fact grader passes them all.

Predict first.

Answer A written for this site

“I’m so sorry it’s late again; that isn’t good enough. Order 1042 ships today and arrives Thursday.”

PASS has the key fact

Owns the problem, then gives the facts.

The toy policy is a choice among answers written for this site, not a language model. Its updates run live in your browser.

The key-fact grader runs 10 rounds on these drafts. What happens to the warm draft, A?