Before and after is not proof
Comparing the month before a change with the month after mixes the change with everything else: season, prices, campaigns, competitors, even the weather. A rise after a change does not mean the change caused it.
The fair way: compare at the same time
Show the change to half of the visitors and keep the other half on today's version, during the same days. Everything else affects both halves equally, so the difference between them is the effect of the change. This is an A/B test.
Measure money, not clicks
A bigger button can get more clicks and fewer orders. Decide up front which number counts: orders, revenue per visitor, average basket, sign-ups that pay. Track the ones that could get worse too, such as returns.
Turn the result into a yearly value
If revenue per visitor rises by 2% and the page earns €500,000 a year, the change is worth about €10,000 a year. A range is more honest than one number: "between €4,000 and €16,000" tells you how sure the result is.
When you cannot run a test
Some changes reach everyone at once, such as a new price list. Compare with a similar group that did not get the change (another region, another product line), or with what the trend predicted. These methods are weaker than a test, so treat their results with more caution.
A simple checklist
- One main number, chosen before the start
- Both versions running at the same time
- Enough visitors, in whole weeks
- A yearly value with a range
- A decision written down: keep, roll back or test again