A/B testing, also known as split testing or bucket testing, is a process of comparing two versions of a webpage or app to see which one performs better. With the use of data and statistics, this method validates new design changes and improves conversion rates.
In product management, this method would involve testing design changes of a webpage against its present design by showing these two versions to similar visitors at the same time. Half of the visitors sees version A and the other half, version B. The team picks the design that drives better results.
The Importance of A/B Testing in Product Management
A/B testing can help product managers create the right things and validate their assumptions. This process is a great way to test an idea. Do take note though that while A/B testing is an effective way to choose which one of two designs is best to launch, this process should not be used for coming up with new ideas. For this, gathering customer feedback should be performed. Specifically, A/B testing is vital in increasing your website’s current metrics by launching the preferred version to your market. As these webpage changes are rolled out, you can expect these to have a positive impact on how your existing customers decide on purchasing a product or service, join email newsletters, as well as other metrics your webpage focuses on.
How A/B Testing is Done
You can test almost anything on a webpage that your site visitors interact with, such as headlines, webcontent, hyperlinks, call to action buttons, images, and social sharing options. If you attend product management training, you will see that using the scientific method would be most effective. This would involve asking a question, doing background research, developing a hypothesis, determining the number of site visitors, analyzing data, drawing conclusions, and reporting the results.
Case in Point
For instance, a product manager for an e-commerce site who has observed that their mobile shoppers are abandoning their online cart at a higher rate than the industry’s standard bounce rate would ask why their mobile customers are abandoning the cart when they reach this page. Next step is to perform a background research. This could be done by checking Google Analytics to check the trend and see if there are similar issues on other webpages. A thorough research would give the product manager confidence as they formulate a hypothesis, such as concluding that the reason why their mobile shoppers are abandoning their carts is because the checkout button is positioned under the screen fold. For the A/B testing, two versions are prepared. Version A is the current design where the checkout button is positioned below the screen fold, and version B is the new design that has the checkout button positioned at the top of the webpage. The product manager can split the site visitors 50/50 and run the test with an A/B testing software. After analyzing data, the PM then concludes that their conversion is twice as high with version B than it was with version A. Lastly, the results are reported to the team and the page design is updated.