TinkerPlan: lookups high, multi-step high, distractions high. 9 of 13 right, cost 2560.
Thinking recorded
The question is how long standard customers have to ask for a refund. The passage says: within 30 days of delivery. Is there a different rule for enterprise? Yes, but the question is about standard customers, so it does not apply. Answer: 30 days of delivery.
Chapter 7 of 7
Workbench: an effort per request
Everything is on the bench: three kinds of request, four effort levels, and the recorded answers graded live. The support lead needs one setting per kind before the assistant goes back on duty.
This chapter: Which effort should each kind of request get?
You will learn
- How to read accuracy, cost and wait together
- Why the best plan mixes efforts instead of using one everywhere
- Why more than one plan can be equally good
Words you’ll meet (2)
- Effort plan
- One effort level for each kind of request.
- Cost
- Here, thinking and answer tokens added up, in relative units: one output token is 1.
The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.
Step 1 of 2
The proposal on the table: high effort everywhere, to be safe. The bench reads every recorded answer for your plan and grades it live. The best plan gets the most answers right (12 is the most any plan gets) at the lowest cost. Cost is thinking and answer tokens, in relative units. Set an effort for each kind and find a best plan.
9 of 13 right · cost 2560 · average wait 5.4 sBEST IS 12 FOR 1249
| Question, effort | Graded | Cost |
|---|---|---|
| How long do standard customers have to ask for a refund? (high)Even at high effort a lookup gets short thinking, and the same answer. | right | 110 |
| What is the cutoff time for same-day shipping? (high)High effort, short thinking: the model saw nothing to work out. | right | 127 |
| How long is the warranty on the dock door sensor? (high)Checks the passage twice and lands where no thinking did. | right | 137 |
| Who leads the Customer Success team? (high)High effort spends a few tokens checking and changes nothing. | right | 102 |
| A customer orders 12 Kestrel K2 scanners and 60 label rolls on one order. What is the total before tax and shipping? (high)Right, and checks its sum a second way: more tokens for the same answer. | right | 327 |
| A customer orders 55 Kestrel K2 scanners as two order lines, one of 30 and one of 25. How much more do they pay than if all 55 were on one line? (high)Right, checked a second way. | right | 344 |
| A customer wants 30 Kestrel K3 scanners. How many can ship today, and how many must wait for Friday? (high)Right; the extra thinking adds checks, not accuracy. | right | 195 |
| An order takes 22 Heron shelving bays from Northfield and 10 from Eastgate. Which warehouses must reorder? (high)Only the longest thinking catches that the rule is about what is left. | right | 178 |
| An in-stock order of engraved Wren label printers is placed on Monday at 10 a.m. Central, with standard ground delivery. What is the earliest day it can arrive? (high)Talks itself out of the question asked: the safest day, not the earliest. | wrong | 259 |
| A customer orders an in-stock dock door sensor at 3 p.m. Central on a Friday, with expedited delivery. What day does it arrive? (high)Right; the extra checks find nothing new. | right | 210 |
| An order has 3 Wren label printers and 2 dock door sensors. How many items are in the order? (high)The longest thinking chases every detail in the note and loses the count. | wrong | 222 |
| How many scanners does this customer have today? (high)Turns a count into a capacity sum because the note invited it. | wrong | 182 |
| Picking for an order starts on Monday morning and ends on Wednesday evening of the same week. How many business days of picking is that, counting Monday? (high)Applies last week’s holiday to this week: more thinking found a detail to misuse. | wrong | 167 |
Token counts are this site’s tokenizer’s; the wait assumes 40 tokens a second after 0.5 s, larkspur’s assumption, not a provider figure.