TinkerA bigger Buy button on checkout B.
Chapter 3 of 7
How many shoppers?
Next idea: a bigger Buy button. The team expects a small lift, about one point, and wants an answer within a week.
This chapter: How many shoppers does a test need to find a small change?
About this chapter
You will learn
- Power: the chance a test finds a real change
- Why small changes need many shoppers
- Sample size, planned before the test starts
Words you’ll meet
- Power
- The share of runs in which a test finds a change that is really there.
- Underpowered
- Too few shoppers to find the change you care about most of the time.
- Sample size
- How many shoppers each version needs, worked out before the test starts.
Step 1 of 5
In this simulation the bigger button truly adds 1 point. Run one week and read the scoreboard.
Run one week.