Tinker Synthetic data

by Samy (Kondasamy Jayaraman) · drawn with Colorfigures by April Zhu

One generation trained on this recipe: 42 documents.

temperature 1, sieve off, 0% real · loss 4.56 · rare words guessed 19.3%

Chapter 7 of 7

Workbench: a data recipe

The team must pick one recipe for the next model: a temperature, a sieve and a share of real data. Each was trained offline.

This chapter: Which recipe stays closest to real text and still guesses rare words?

About this chapter

You will learn

  • How each choice moves loss and rare words
  • Why real data matters most
  • Why more than one recipe can be best

Words you’ll meet

Recipe
A temperature, a sieve and a share of real data.
Real share
How much of the training text is Larkspur’s own documents.
Workbench
The chapter’s free-play bench: change the recipe and see every result.

Step 1 of 2

The first is copies only. A recipe wins when none other has lower loss and more rare words guessed.

Change it on the .

Loss 4.56 · rare words guessed 19.3%BEATEN

Your recipe, as trained offline
Recipetemperature 1, sieve off, 0% real
Training documents42: 42 model-written, 0 real
Held-out loss4.56 (Larkspur mini: 4.04)
Rare words guessed19.3% (Larkspur mini: 27.1%)
Beaten bytemperature 0.7, sieve off, 50% real: loss 4.37, rare 23.5%

Larkspur’s toy: Larkspur mini and copies of it, trained offline on Larkspur’s documents. It shows the mechanism, not a production model’s numbers.

Recipe