What we agree before launch

We describe one change, choose a primary measure and estimate how many visitors we need. For example, will putting the prices side by side help shoppers choose between subscribing and buying once? An order counts. An extra click is a clue.

You get the test plan, working variants, a check on traffic allocation and the analysis. We'll also check returns or order value if they matter to the decision.

When does the test stop?

With a fixed-sample test, we agree on an end point in advance. A green dashboard on day two isn't a reason to stop. If the evidence is inconclusive, we say so and decide whether another test is worth running.

What if traffic is low?

A broken size filter doesn't need an A/B test. For open questions, observing a few customers can get you further. The worked sample-size example shows what a change from 2.0 to 2.4 per cent means for test duration.

At SWISSpur, the add-to-cart rate rose by 44.99% relative to control. See exactly what we changed above the purchase button.