Tinker Diffusion models

by Kondasamy Jayaraman · drawn with Colorfigures by April Zhu

256 × 256 px: 65,536 positions × 50 steps = 3,276,800 units of work.

Chapter 6 of 7

Why it costs what it costs

Larkspur’s catalogue has thousands of parts, and finance wants to know what a picture costs before anyone presses go.

This chapter: What makes a diffusion picture expensive, and how do newer models cut it?

About this chapter

You will learn

  • Why the work grows with steps and with picture size
  • How latent diffusion shrinks what the denoiser reads
  • How few-step models cut the steps instead

Words you’ll meet

Position
One place in the picture the denoiser reads each step: a pixel, or a cell of the latent.
Latent diffusion
Running the denoiser on a small encoded copy of the picture, then decoding the result to full size once.
Few-step model
A model trained to make a good picture in one or a few steps, often distilled from a diffusion model.

Step 1 of 4

Each step, the denoiser reads every pixel of the picture. Larkspur’s toy prints are tiny; catalogue images are not.

Predict first.

Double a picture’s width and its height. How much more work is each step?