TinkerThe run with one document copied 10 times into its data.
Chapter 4 of 7
Data is the curriculum
The team reran training with one document, Larkspur’s environmental policy, copied 10 times into the data.
This chapter: What does a model learn from text that repeats?
About this chapter
You will learn
- Why the data decides what a model learns
- How a repeated document gets memorised and parroted
- Why teams remove duplicates before training
Words you’ll meet
- Data mix
- Which documents go into training, and how often each one appears.
- Duplicate
- The same text appearing more than once in the training data.
Step 1 of 4
Same model, same steps; only the changed. One document, 10 copies in all.
Predict first.