A/B testing, or split testing, is the secret to a truly powerful website marketing or online strategy. Where website design can sometimes feel like a shot in the dark, A/B tests are a scientifically-backed technique that can actually prove what truly works and what doesn’t, in the real world.
As with any experiment, A/B tests require you to follow procedures in order to see meaningful results. Here are the three simple steps we follow to build powerful split tests that inform the marketing campaigns we run for our clients. This is our quick guide to our favorite form of website conversion optimization: A/B testing.
1. Isolate One Variable and Control the Rest
The problem with most marketing analytics is that they attempt to answer too many questions at once. They leave too much to assumption. A proper A/B test isolates a single variable while you control all others. By keeping all other variables identical and isolating just one element of your campaign, you can more precisely and accurately attribute any difference in results to that particular factor. Be smart here – there are more factors than meets the eye. If you’re split teseting an email marketing campaign, for example, don’t forget to send out both variations of your email on the same day at the same time.
2. Formulate a Hypothesis
In order to select the variable of your split test, you need to make an educated guess as to which factor might produce the strongest results. Then formulate a hypothesis as to what results you believe will be influenced by your test, how they will be affected, and why. This will help keep your experiment focused and guided. By predicting results, you’ll be able to work backwards to select the best variable to test.
3. Test on a Large Enough Sample
The larger the sample size, the more accurate your results will be. The caveat here, however, is that if you use too big a sample size, you risk presenting too many people with the weaker version. A general rule of thumb is that you need at least 1,000 people in your sample to make a split test worthwhile. From there, a split test would start by sending Version A to about 25% of your audience and Version B to 25% of your audience. The winning version would be sent to the remaining 50% of your audience. As the number of people in your sample increases, you can slowly cut back on the percentage you expose to your split test.