Module 16 · Multimodal
Images become tokens too: small squares of pixels, counted and priced like text. Here you cut an invoice into patches, shrink it until the small print goes, read tables and handwriting two ways, and pick a method and a size for each kind of document.
The whole idea in one example
180 image tokens
An invoice, probably from Brightwater Fasteners. The number looks like INV-20411 and the date reads 14 Sep 2026. Total $1,284.50. The order number and the small print are too small to read.
At 400 px only the biggest print survives. The invoice number’s last digit is a guess, and a wrong one.
2,610 image tokens
An invoice from Brightwater Fasteners to Larkspur Supply Co.: INV-20417, dated 14 Sep 2026, for order PO-5531. Three lines: brass hinges, BR-300 shelf brackets and a box of wood screws. Total $1,284.50, payment terms Net 30.
Every field right, small print included.
Both readings were written for this site to illustrate the cited findings; they were not measured on a real model. The fields are graded live by the Evals module’s key-fact check, and the token counts follow Anthropic’s published patch rule.
How it works
Cut into 28 px patches, one token each
Claude views images in patches instead of pixels: each patch is a 28 × 28-pixel block, one visual token, so an image costs ⌈width / 28⌉ × ⌈height / 28⌉ tokens.
Source, checked 21 October 2026
Fewer pixels, fewer tokens, and the small print goes first
Anthropic advises that important text in an image should be legible and not too small, and warns that resizing a large image can make its text less legible.
Source, checked 21 October 2026
OCR loses tables and handwriting; vision keeps the layout
OCRBench, a 2023 study of large multimodal models reading text, found weak spots in multilingual text, handwritten text, non-semantic text and mathematical expressions.
Source, checked 21 October 2026
Grade every field against a reference, not one overall score
When one angle can’t settle it, ask for another
Anthropic’s listed limitations: Claude may make mistakes on low-quality, rotated or very small images (under 200 pixels).
Source, checked 21 October 2026
Check what was read; text in an image is data, not orders
OWASP lists multimodal injection under prompt injection: an attacker hides instructions in an image that accompanies harmless text, and the hidden prompt changes what the model does.
Source, checked 21 October 2026
The documents and photos are drawn in code and rasterised in your browser at the size you pick. The readings were written per image, size and method to illustrate the cited findings; the grading and token counts run live.
The story
Larkspur Supply Co.’s assistant starts receiving pictures: supplier invoices, packing slips, notes from the warehouse and photos of broken parts. You’re on the team that decides how each kind of image is read, and checks what comes back.
Tokens showed how text becomes tokens; here pictures do. The workbench’s best plan is one of 3 that tie.
Your path
Chapter 6 · done
How can an image make the assistant wrong, or make it do something it shouldn’t?
Chapter 7 · done
Which method and size should each kind of document get?
Product managers, engineers and analysts sending documents and photos to a model, and paying for them. Tokens comes first; no code or maths needed.