Tinker Multimodal models

by Kondasamy Jayaraman · drawn with Colorfigures by April Zhu

OK

PASS1,600 px: 2,610 image tokens; 5 of 5 fields read right.

Chapter 2 of 7

Size and detail

An invoice at full size costs thousands of tokens. Larkspur gets hundreds of invoices a week, and someone suggests sending them smaller.

This chapter: What do you lose when you send a smaller image?

You will learn

  • Why fewer pixels means fewer patches, and so fewer tokens
  • Why small print is the first thing to go
  • Why a very large image is scaled down before it is read
Words you’ll meet (3)
Resolution
How many pixels an image has across and down. Here the size is the long edge in pixels.
Downscale
Shrinking an image to fewer pixels. The model’s host does it to any image over its limits.
Small print
The smallest text on a page: terms, footnotes, fine detail. It blurs away first.

The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.

Step 1 of 3

At 1,600 px the invoice costs 2,610 image tokens, and the reading gets every field right. Fewer pixels means fewer patches, so fewer tokens. Before you shrink it:

Accounts wants invoices sent smaller, to save tokens. Shrink this one step by step: which field becomes unreadable first?