A/B Testing in 7 Steps to Quickly Improve Your Conversions

Why Should You Do A/B Testing?

A/B testing is a powerful tool for businesses, whether you want to increase Web-site conversions or improve the return on investment (ROI) for your marketing campaigns.

There are many advantages of A/B testing, but some of the most notable ones include the following:

• allowing you to test different hypotheses to see what works best for your business
• enabling data-driven decisions rather than relying on gut instinct
• helping to avoid the sunk-cost fallacy by investing only in what works
• making it easy to track and measure results
• reducing bounce rates
• minimizing risks and making low-risk modifications for maximum profit
• increasing sales and profits
• improving your brand’s image and increasing goodwill in the marketplace
• increasing user engagement by optimizing email subject lines or social ad copy
• identifying the pitfalls of your landing pages
• eliminating guesswork by driving real-time data

Let’s look at an example: Say, you have a Web site and want to get more people to click your home page’s primary call to action (CTA). You could test different variations of your home page’s headline, copy, or CTA button to see which one works better. Figure 1 shows two versions of a Web page in which the color of the CTA button is different.

Image source: Adoric

Let's assume that the second version gets 20% more clicks than the first version, and the same number of visitors are coming to both versions of the page. This suggests that the green CTA button is more effective for this particular Web site.

Note—Read “7 Basic Best Practices for Buttons,” by Caroline Jarrett.

In a similar way, you can test the effectiveness of different headlines or Web site copy.

Running A/B tests can have significant positive impacts on not only your Web site but your email-marketing campaigns, social-media ads, ecommerce sales, gaming apps, and more. But only if you do it correctly. For example, a popular, classic gaming site tested higher resolution card images in their games and found that the number of games played per user increased by 11%.

The Different Types of A/B Testing

You can use A/B testing for a variety of purposes, from testing the effectiveness of a headline to see which one gets more clicks to testing which color of button users are more likely to click. There are several different types of A/B tests, each with its own advantages and disadvantages. Let’s consider some of the most popular types of A/B testing.

Split Testing

Split testing is a method of marketing experimentation in which you show users two versions of a piece of content at random, then keep the version that performs better, as depicted in Figure 2.

Image source: Optimizely

It’s best to use split testing when you want to test two versions of something that you can’t test at the same time, such as a landing page or an email newsletter’s subject line.

Advantages:

• Can provide clear results about which version is better.
• Doesn’t require a lot of traffic to get reliable results.
• Is ideal for testing changes other than user-interface changes.

Multivariate Testing

Multivariate testing is a method of marketing experimentation in which a company shows multiple versions of a piece of content to users at random, then uses the version that performs the best going forward, as depicted in Figure 3.

Image source: HubSpot

It’s best to use multivariate testing when you want to test different combinations of variables simultaneously on a Web page or in an email message.

Advantages:

• Saves time that you would otherwise spend on multiple, sequential split tests.
• Lets you test the impact of multiple elements such as a hero image, CTA button, and headline together.

Multipage Testing

Multipage testing is a method of A/B testing that lets you test multiple pages of your Web site at the same time, as depicted in Figure 4. You can use this type of testing to test changes to your home page, product pages, category pages, or any other type of page on your Web site.

Image source: Aillum

Multipage testing is a great way to see how a variety changes impact your overall conversion rate. By testing multiple pages at once, you can see how changes to one page affect other pages on your site. This can help you optimize your Web site for conversions and improve the user experience.

Advantages:

• Test multiple pages at the same time.
• Assess how changes to one page affect other pages on your site.
• Create consistent experiences for your target audience.

7 Essential Steps of Conducting A/B Testing

Now that you understand the importance of A/B testing, you should also know that doing A/B testing wrong can be a waste of time and money. You must learn how to conduct A/B testing in the right way. When conducting A/B testing, carry out the following steps.

1. Start with a Hypothesis.

Before you start your A/B test, you need to have a hypothesis—that is, a prediction or an educated guess about what would happen if you make a certain change to your Web page, application, or ad campaign. For example, if you want to test different headlines for a blog post, your hypothesis might be: “The headline with the keyword X will perform better than the headline without it.” Creating a hypothesis is important because it helps you focus your A/B test and makes it easier to measure the results.

2. Choose Your Goals and Metrics.

The next step is to choose your goals and key performance metrics. You need to know what you want to achieve with your A/B test and how you’re going to measure the results.

Some common goals for A/B tests include the following:

• increasing conversion rates
• getting more clicks
• increasing user engagement
• reducing bounce rates

Plus, some common metrics that you can use to track your progress and determine whether your changes are having the desired effect include the following:

• total amount of traffic
• numbers of unique visitors and repeat visitors
• conversion rates
• click-through rates (CTRs)
• amount of sales generated
• engagement rate
• numbers of likes, comments, clicks, or shares
• bounce rates and dwell time
• cost per lead

You should select your key performance metrics based on your end goal because they determine how you measure the success of your test.

3. Create a Control and a Treatment.

Once you’ve chosen your goals and metrics, create a control and a treatment. The control is the original version of your Web page, application, or ad campaign. The treatment, or variation, is the new version you’re testing. It is important to change only one thing at a time in your treatment so you can be sure that the results are due to the change you’ve made. For example, if you’re testing different headlines, the control would be the original headline and the treatment would be a new headline.

Don’t try to test too many things at once. If you test multiple changes, you can’t tell for sure which change caused an improvement or decline in conversions. For example, if you change the headline and the banner layout at the same time, you won’t know which of the two changes in a variation helped drive more clicks. Figure 5 shows an A/B test with an ideal control and variation, or treatment.

Image source: Wingify

When conducting A/B testing to improve your conversions and bottom line, some of the best elements to test include the following:

• calls to action (CTAs)
• headlines
• copywriting, fonts, and writing styles
• pop-up messages
• email-newsletter subject lines
• images
• videos
• landing-page designs and layouts of elements such as CTAs, headlines, and featured products or services
• email design elements such as color, fonts, and layout
• text fonts and style
• form length
• color
• types of social proof such as ratings, reviews, and testimonials, as well as their placements
• navigation
• search filters

4. Run Your Test.

Once you’ve created your control and treatment, it’s time to run your A/B test. You need to determine how long the test should last. It is important to run a test long enough to get accurate results, but you don’t want it to last too long because that could be expensive and time consuming.

A/B tests require a statistically significant sample size. Set up your A/B test so you show the original version to half of your visitors and the new version to the other half. You also need to determine how many visitors you need to test. This depends on your goals, metrics, and the amount of traffic you typically get on your Web site. Once you’ve determined these parameters, you can start your test.

5. Evaluate the Results.

Once you’ve run your test, you must evaluate the results. Compare the performance of your control and treatment to see whether there was a statistically significant difference.

Track your results over time to see which version performs better. If you observe a significant improvement in conversions for the new version, make it a permanent change. If not, go back to the drawing board and test another design. If there were a significant difference, you could implement the treatment on your live Web site. If there wasn’t a significant difference, you could try a different treatment or keep the control version. A/B testing is a powerful tool that can help you improve your Web site, application, or campaign.

6. Use an A/B Testing Tool.

To get accurate A/B testing analytics and insights, you should use tools such as HubSpot, Google Analytics, or Optimizely. These platforms can help you track Web-site visitors more easily and efficiently and better understand how visitors respond to A/B testing.

For example, if your goal were to increase the number of qualified leads for your business, you could use a platform such as Salespanel for lead generation and lead tracking. This tool can also help you with everything from real-time, Web-site tracking to UTM (Urchin Tracking Module) tracking, Gmail and Outlook email tracking, and newsletter tracking.

There are also tools that let you sync important data between apps and organize it into visual reports and exportable Excel spreadsheets. One example is Coupler.io, which lets you link Salesforce to Excel and export data from HubSpot, QuickBooks, and other tools to Excel, Google Sheets, or BigQuery.

7. Leverage User Segmentation for Targeted Testing.

A/B testing with your entire audience can give you an idea about generic trends, but might not be the most accurate way of measuring the impacts of your design changes. This is where user segmentation comes in.

Audience segmentation is the process of dividing your users into groups based on their characteristics such as age, gender, location, and interests. Segmentation lets you target your A/B tests to specific segments of your audience and get insights into specific trends. For example, you might want to segment your audience by location and test different versions of your Web site with users in different countries. This would let you see how your changes impact users in different areas.

Audience segmentation is a powerful tool that can help you get more accurate results from your A/B tests. Figure 6 shows a few common types of market segments, or user groups, that you can leverage for targeted A/B testing.

Image source: Pinterest

You could also use account-based marketing and target specific accounts when conducting A/B tests. This is a great way to test changes that are specific to enterprise customers. You should leverage this approach to ensure your B2B (business-to-business) success.

By following all these steps, you can ensure that you’re conducting A/B testing in the right way.

Key Takeaways

A/B testing is an essential element of any conversion-optimization strategy. By conducting A/B tests, you can determine which elements of your Web site are most effective in converting visitors into customers.

In this article, I’ve provided a step-by-step guide to conducting A/B testing on your Web site. I’ve also shared some tips for making the most out of your A/B-testing efforts.

A/B testing is a powerful tool, but it’s only one part of the conversion-optimization puzzle. If you want to take your conversions to the next level, try combining A/B testing with other optimization strategies such as user-experience testing and personalization.

When conducting A/B testing, there are a few things you need to keep in mind, as follows:

• Set up your test correctly from the start.
• Make sure you have enough traffic to get accurate results.
• Test one element at a time.
• Run your tests for a sufficient length of time.
• Be prepared to make changes to your Web site based on your results.

If you follow all these tips, you’ll be well on your way to conducting successful A/B tests that improve your conversion rates. Good luck! Do you want to learn more about how to conduct A/B testing? Ask your questions in the comments below.