TinkerLook 1: a screenshot of Home, 1,334 tokens.
Chapter 1 of 7
Look, then click
Brightwater’s trade portal has no API, only web pages. Larkspur gives an agent the job of re-ordering hinges there, the way a person would.
This chapter: How does an agent use a website made for people?
About this chapter
You will learn
- Why the agent sees only screenshots
- How a click comes back as a point
- Why every step starts with a fresh look
Words you’ll meet
- Host
- The program that runs the computer for the model: it takes the screenshots and does the clicks.
- Screenshot
- A picture of the screen, taken by the host and sent to the model as an image.
- Agent loop
- Look, act, look again, repeated until the task is done.
Step 1 of 4
The took a of Brightwater’s portal and sent it to the agent: re-order the hinges.
Click where the agent will click.