A/B Testing for Landing Pages

A very popular way to test the effectiveness of a landing page is to use an A/B Test. The idea behind A/B tests is very simple. You create two versions of the same landing page, however, one version of the page has an alternative variable that you want to test. That variable can be a heading, an image or even the color of the submit button on the form. The purpose of the A/B test is to run both versions simultaneously, alternating visitors to each version and comparing the results of both versions. It is important that you only test one variable at a time, otherwise you will not know what aspect of the A/B test contributed to the difference in conversion, which is what you should be testing.

There are dozens of tools which can be used to help with an A/B test, Optimizely and Visual Web Optimizer (VWO) are two popular choices. These tools automatically redirect the visitor randomly to either version A or version B of your landing page. The tool will also help you measure the outcome of your test. If one version performs significantly better than the other version, you should change to using the winning version of your A/B test.

You should notice how we said “significantly better” in the last paragraph. This is because some variations in conversion are not “statistically significant”. Statistical significant is a way to determine whether the result of the test is due to the change you made, or due to pure chance.

Testing for statistical significance is not difficult. You need to know the sample size or the number of people who saw both landing page versions; same variation or type of people that saw both landing pages and target significant level which refers to the “p-value” which is the confidence interval. There are variety of online calculators which will tell you whether the results of your A/B Test are statistically significant, here is one such calculator by HubSpot. If you want to learn more about the math behind statistical significance in A/B Tests you should read HubSpot’s guide to statistical significant.

In some cases, you will not have enough visitors or enough data to determine the successful landing page. In this circumstance you should continue running the rest until your landing pages receive enough visitors for analysis.