Tinker Synthetic data

by Samy (Kondasamy Jayaraman) · drawn with Colorfigures by April Zhu

“Q: Does the Kestrel K3 come with a charger? A:”

Chapter 3 of 7

The sieve

Twenty replies came out, some repeated, some off topic. Before any go into training, the team runs them through a sieve.

This chapter: What does a sieve keep, and what does it throw away?

About this chapter

You will learn

  • How dedupe drops copies and near-copies
  • How a fact filter keeps replies that name a product
  • Why a sieve can only remove, never add

Words you’ll meet

Sieve
Checks that drop written examples before training.
Dedupe
Dropping a reply that copies, or mostly overlaps, one already kept.
Fact filter
Keeping only replies that name one of Larkspur’s products.

Step 1 of 5

Larkspur mini answers “Q: Does the Kestrel K3 come with a charger? A:” at temperature 0.7. Some replies repeat, some wander.

Write twenty at 0.7.

Loading Larkspur mini…