TinkerThe phrase: “ safety boots and high-visibility”
Chapter 1 of 7
One word at a time
A vendor promises a model that writes 1,009 tokens a second. Larkspur’s team wants to know what that speed buys.
This chapter: How does a model write a reply, and what does each token cost?
About this chapter
You will learn
- Why a left-to-right model takes one pass per token
- What a pass is
- Why passes set a model’s speed
Words you’ll meet
- Token
- A piece of text a model reads and writes: a word, part of a word, or a mark.
- Pass
- One run of the model over everything it can see. Each pass takes about the same time.
- Larkspur mini
- Larkspur’s tiny left-to-right model from the Transformers module. It writes one token a pass.
Step 1 of 3
, the typewriter, continues a phrase from the safety handbook. Each is one run of the model.
Predict first.