TinkerA supplier’s invoice, about to be read.
Chapter 1 of 7
A picture is tokens too
Larkspur’s assistant answers questions from text. Now suppliers email invoices as images, the warehouse photographs its notes, and customers send pictures of broken parts. The team wants the assistant to read them.
This chapter: How does a model that reads tokens read a picture?
You will learn
- Why an image is cut into small squares called patches
- How each patch becomes one token, like a piece of text
- How to count an image’s tokens from its width and height
Words you’ll meet (3)
- Multimodal model
- A model that reads more than one kind of input, such as text and images.
- Patch
- A small square of an image, 28 × 28 pixels here. The model reads each patch as one token.
- Visual token
- The token a patch becomes. Image tokens are priced and counted against limits like text tokens.
The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.
Step 1 of 3
Larkspur’s assistant reads tokens: the small pieces of text a model reads and writes (the Tokens module covers them). A multimodal model also reads images. Before you see how: