The short answer
Decide the length before you start. It comes from three numbers: how often visitors convert today, the smallest improvement worth detecting, and how many visitors the page gets each day. Then round up to whole weeks, with at least one full week.
Our free A/B test calculator does the maths: a shop converting 3% that wants to detect a 10% improvement needs about 53,000 visitors per version.
Why whole weeks
People buy differently on Monday morning and Saturday evening. A test that runs Tuesday to Friday compares weekday shoppers only. Full weeks give every version the same mix of days.
Smaller changes need much more traffic
Halving the improvement you want to detect needs about four times the visitors. If the calculator says your test needs months, test a bolder change, a busier page or fewer versions.
Don't stop at the first good day
Checking results every day and stopping as soon as one version looks ahead is the most common mistake. Early leads come and go by chance, and stopping on one turns luck into a fake winner. Either wait for the planned number of visitors, or use a method designed for daily checks. Verdicto uses such a method, so results stay valid when you look every day.
Other reasons to run a bit longer
- Big seasonal moments (sales, holidays) in the middle of the test
- Products bought rarely, where customers think for days
- Changes people need to get used to, such as a new menu