TinkerThe toy policy over four written drafts, before any training.
Chapter 4 of 7
Grade, don’t judge
Some questions have one right answer. A warranty’s length is in the documents, so a program can check every draft, with no people needed.
This chapter: How does a model learn from a grader that checks its drafts?
About this chapter
You will learn
- What a checkable grader is
- How GRPO scores each draft against its group
- Why each step is clipped
Words you’ll meet
- Grader
- A program that checks a draft and gives it a reward: here, Evals’ key-fact check.
- Group
- Several drafts the model writes for the same question, graded together.
- GRPO
- Group relative policy optimization: each draft is scored against its group’s average, and the model leans toward the above-average ones.
- Advantage
- A draft’s reward minus the group’s average, divided by how spread out the rewards are.
Step 1 of 5
One right answer, so a program can check it: a . The model writes a of drafts.
Press Draw a group.
The customer asks
“How long is the warranty on the DS-100 dock sensor?”
The toy policy is a choice among answers written for this site, not a language model. Its updates run live in your browser.