TinkerLarkspur mini, ready to guess the next piece of text it never trained on.
Chapter 2 of 7
Honest jars
Before trusting the assistant’s numbers, the team checks a model whose every guess can be counted: Larkspur mini, this site’s tiny model.
This chapter: Is a tiny model honest about how sure it is of its own guesses?
About this chapter
You will learn
- How a model’s odds for the next piece are a confidence
- Why held-out text is the fair test
- How to read a reliability diagram
Words you’ll meet
- Top guess
- The next piece of text the model gives the highest probability.
- Held-out text
- Documents the model never trained on, kept aside to test it fairly.
- Reliability diagram
- A chart of stated confidence against how often the answers were right. Honest models sit on its diagonal.
Step 1 of 4
Larkspur mini gives every possible next piece a probability. Its , and the probability it gave, are one marble.
Predict first.