76% of Google and Facebook budgets are wasted.
A/B testing can help you reduce that waste. No amount of research and statistics can point you to the creation of the most effective campaign right from its inception. Not to mention it takes time for any algorithm to “warm up”. Optimizing your marketing and sales funnel as you look to achieve the best conversion rate possible is a best practice that you should be incorporating in every aspect of your business.
What is A/B Testing?
If you’re not A/B testing, it’s likely that you’re wasting your money. In 2018, Invesp reported that 71 percent of companies conduct two or more A/B tests every month. These tests are conducted on their corporate websites, landing pages, e-mails and paid search campaigns.
A great way to understand A/B testing is to think of it like organizing a tournament, kind of like what goes on during March Madness, to figure out which version of an ad, landing page, sales copy or other promotional efforts best converts your audience into clients. Let’s look at the example of an online ad.
You create two different versions of the ad while targeting the same audience. You allow the two ads to run on an advertising platform, keeping an eye out for which one performs better. Once you have enough data to arrive at a conclusion, you eliminate the least effective ad and replace it with another challenger that runs in its place. Eventually, your testing should give you a near-ironclad advertisement that has the most effective sales copy and visual media attached to it.
How Does A/B Testing Work?
This method of testing works by giving you an actionable and measurable way of figuring out what your audience really wants as well as what the various algorithms that govern online advertising are looking for. If you think of creating an effective ad as a process akin to being placed in a pitch-dark warehouse and asked to find a small object, A/B testing is like being able to box off the parts of the warehouse that you’ve already searched so that you know which space you’ve covered.
Even though you may know your product or service very well, attracting potential customers is not as easy as sharing a feature-rich sales pitch with them. You need to figure out how your prospective customers think. A/B testing gives you the ability to systematically understand how users are best pitched to and to what they respond to.
What Are the Benefits of A/B Testing?
There are two main benefits to running an A/B test and they are both equally significant in contributing to a sustainable business.
Get More Out of Your Marketing Budget
When you A/B test, you are creating more and more efficient marketing. Let’s use the example of a paid ad again. Let’s assume you are running an ad in an attempt to sell a 100 dollar product. If you have to spend 100 dollars on ads in order to make a sale, you are ending up with a loss. When you subtract your ad budget and operational costs, not to mention the value of your time spent, from the money you made, you are putting in more than you are getting out.
Now, without changing any of the variables except for the effectiveness of your ad, let’s assume that with that same 100 dollars you are spending on your advertising, you improve your ads so that you make 2 sales. Automatically you have made your advertising campaign viable. You now know that every 100 dollars you put in will get you 2 sales on average. How about selling 4 units of your product with the same ad budget?
Dominate Your Niche
The second benefit of A/B testing is the fact that it helps you gain on your competition. In the example mentioned above, the lower you can get your marketing costs, the more aggressively you will be able to compete against the other market players in the online space. If you are paying less per conversion, you are automatically putting your competitors at a disadvantage.
Landing Page Testing
Taking this concept a little further, it is important to understand how a strong online presence is built. Search engines such as Google will rank your content based on how relevant and appealing it is for your chosen audience. Search engines have algorithms that rank websites based on how internet users respond to the content they interact with.
For example, you have a website for your product and users tend to leave your page only a few seconds after arriving. This means that a whole host of metrics that Google tracks, such as bounce rate, average on-page time and pages viewed per visit, will be abysmal.
Bounce rate is the percentage that denotes how often users click through to your page but leave your website without venturing to a second page to see what your business is all about. Average on-page time shows how long visitors stay on your website and is important in Google’s logic because it means that the more time people spend on your internet property, the more they enjoy what you have to offer. The average number of pages viewed further reinforces this point by showing how many pages people are willing to navigate on your website.
If your competitors are keeping their visitors on their landing pages longer and users are flipping through their websites in the search for more information, they will be valued higher by Google and other search engines.
This is where landing page testing comes in. By testing two different versions of your content, you can track metrics to see which one is more effective. You can expedite your results by running ads that send visitors to your two landing page versions.
Also, landing page effectiveness should be viewed in conjunction with the ads that you run. If you create a terrific ad that leads to a horrible landing page, your ad effectiveness will suffer. The opposite is also true. A weak ad that doesn’t manage to send users to a great landing page will be detrimental to your results. You need to consider each part of your marketing as an important gear in your promotional engine.
Should You Be A/B Testing?
In short, yes. Most companies should be using this method of optimizing their marketing. If your website is lacking significant traffic, A/B testing may not be your primary concern and may not yield helpful results. In this case, try communicating with current customers to gain valuable insight into how you can improve your numbers.
Setting Up an A/B Test
Now that you understand the benefits you can obtain from A/B testing, it’s time to go through the process of setting up an A/B test. You should be keeping detailed notes throughout the process so that you can gain maximum insight and start to understand some of the nuances of what works and what doesn’t. This will prove to be a time-saving technique that will pay dividends in the future. Setting up an A/B test for the first time will remind you of the scientific process and rightfully so.
1. Start With Research
The time you devote to research will move your starting point further ahead and make your campaigns more efficient from the get-go. Gather any data you have on your landing page from analytics so that you have a baseline to work with. If you’ve run any similar actions in the past, you’ll want to dig that data up and have it available as a control.
Research should include looking into tools and methods that will make your campaign more effective. Investing in a heat map tool can be a game-changer in your quest to understand how your visitors interact with your landing pages. A heat map will give you a view of how your visitors navigate your website and what elements they deem significant enough to spend time on and which ones they ignore.
Places to start:
- most visited pages on your website
- pages with the highest bounce rate
- key step in the funnel where customers are falling off
2. State Your Hypothesis
You should have a clear goal right from the start that you state as precisely as possible. For example, your hypothesis is that you can run an ad campaign that will achieve a 5 percent click-through-rate with 10 percent of the users that click-through becoming paying customers or signing up for your mailing list.
Your initial hypotheses may be slightly arbitrary but try as much as you can to use data to support the values you are shooting for. Obviously, achieving metrics that are profitable will be a good starting point. Calculate how much you are willing to pay for each click. Then, figure out the number of clicks that are acceptable in order to achieve a conversion; sale, newsletter signup or otherwise.
Understand key metrics:
- What is your current conversion rate?
- Which channels are driving the bulk of your traffic?
- Survey customers to understand what they like and what they would like to see
3. Create the First Variations
Depending on the platform on which you will be advertising, you will have specifications as to what your ad can resemble. Facebook, Google and other networks that offer ad placement have limits on the number of characters of which your copy can be composed. Image sizes have to meet dimension guidelines and be compliant to terms of service as to what is shown.
Once you understand the specifications, create two different versions of your ad. Make sure that the differences concern pivotal variants, meaning that the difference between the two versions will allow you to draw an important conclusion about the future of your campaign. To better illustrate this point, consider your call to action. “Buy Now,” and “Don’t Miss this Offer,” will invoke different responses in relation to your product offering and give you a measurable answer as to which is best for you.
4. Run the Test
It is now time to place your ads to the test and find out which version is better.
Predetermine a sample size and run the test for full weeks, usually at least two weeks. Be sure to factor in any seasonal or periodical factors that may affect your conclusions. Running an A/B test during the Christmas holiday season may yield different results than during mid-August. A typical A/B test can take anywhere between three to four weeks.
How do you predetermine sample size? You can calculate your sample size with tool like Evan Miller’s A/B Testing tool:
In this example, we told the tool that we have a 20% conversion rate and want to detect at least 5% uplift. The tool tells us that we need 1,030 visitors per variation before we can look at statistical significance levels. For practical purposes, know that 80% power is the standard for A/B testing tools.
In addition to significance level, there’s something called statistical power. Statistical power attempts to avoid Type II errors (false negatives). In other words, it makes it more likely that you’ll detect an effect if there actually was one.
5. Evaluate
The evaluation phase is where the magic happens. If you’ve done extensive research and planning, you will get data sets that lead you to actionable measures. Which ad had a better click-through-rate? Which ad had the lower cost-per-click? How many conversions resulted from each ad?
6. Optimize
Since you now have two ads of which one was better at achieving your goal, you can spend some time determining what makes an ad more successful. Apply what you have learned and create a third ad that will harness the power of your newly-attained insights. Run this ad alongside your initial winner and see which one prevails.
7. Record Your Conclusions
Recording the conclusions you derive from this process will not only help you create better ads, but it may just help you improve your brand overall. You will find that users respond better to certain words and phrases and that some images attract their attention more than others. Maybe they find clicking-through to a page with more information about your product to be more helpful than being taken directly to a sales page.
Furthermore, an A/B test can be a valuable product research and development tool. A/B testing can tell you what your prospective customers like and dislike about your product or service so that you can improve your design and production.
Examples of Split Testing You Can Run
Here are 3 actionable examples of tests you can run to improve your results.
1. Testing Your Call-To-Action Button
Colors, shapes and fonts can play an important role in whether or not your website’s visitors click on your call-to-action button. Test a few different options before arriving at the one that maximizes conversions.
The below A/B test showed that a red CTA button outperformed a green one by 21 percent based on 2,000 page visits.
2. Test Your Product Launch Offering
The well-known game company Electronic Arts (EA) ran a split test for the release of its SimCity 5 game. The company created a control version of their pre-order page where they offered a 20 percent discount on a future purchase for those that bought the game and tested it against a page where the game was sold at regular price with the option of upgrading to a deluxe version. The surprising conclusion was that their fans were not interested in the discount. The variation without the offer brought 43.4 percent more purchases. This would suggest customers just really wanted to buy the game, no extra incentive necessary. Customers were anxious to get their hands on the game as soon as possible.
Slit-testing a product launch can help put your product on the right track to maximizing sales. Don’t hesitate to test your current products and services against sales offerings that could perform better than your current marketing.
3. Test Personalization
Imagine I’m interested in buying a necklace for my new girlfriend. I’m not ready to purchase something on my first visit, but, on my second visit the homepage features a new necklace collection while removing any rings, earrings, and bracelets. This will keep me engaged.
Now let’s take it one step further. Imagine that I spend 3 minutes on the site (assume the average page view time is 2 minutes) – this means I’m probably seriously interested in purchasing something, which triggers a coupon code for 20% off and an email with a unique coupon code.
Then I return back to work.
While checking my emails, I see the coupon code and click on the button in the email – that’s when the coupon code is automatically applied. That might seem subtle but if I had to go find the coupon code or feel like I needed to remember my coupon code, I might lose interest and just close the session.
This example shows how you can make the experience simple and effective at the same time.
Additional actionable A/B tests
- test your headlines
- start with your layout
- experiment with copy
- try tweaking your checkout page
- test your forms
What you can expect
A successful test can lead to a 50% improvement in conversion rate.
Mistakes to Avoid
There are some common mistakes you should avoid so that your conclusions and budget are not thrown off. Mistakes can also produce incorrect conclusions that you may not uncover until you’ve already gone through the process.
Mistake #1: Testing Too Many Variables
If your car doesn’t start, a mechanic will attempt to find the cause of breakdown by using the process of elimination. There are plenty of components to examine, from ignition to exhaust pipe. If the mechanic were to start concurrently replacing different parts that determine your car’s operation, you may never know what the cause of breakdown truly is. You won’t know if your car is now less likely to stop working or if changing a part means prolonging the vehicle’s operation by a few weeks.
Similarly, when running an A/B test, your goal is to be as surgical as possible in creating an effective campaign. Choose variables to test for that are easy to isolate and track. If your two variations differ wildly and you suspect that concrete conclusions may not be extracted from the process, simplify the ads. Focus on aspects that are important for users and tweak as necessary.
Mistake #2: Not Balancing Traffic
Driving adequate amounts of traffic necessary to draw conclusions is key to a successful A/B test. A small amount of clicks may not be statistically significant for deciding whether the action is successful or not. Also, sending similar amounts of traffic to both versions is important for making your choice between the two.
Mistake #3: Incorrect Campaign Duration
Not only will you have to run your campaign for a duration that is appropriate for attracting enough visitors to drawing your conclusions, but you will also have to account for the context of your campaign. We mentioned seasonality earlier and it is quite relevant here. Your data needs to reflect a uniform set of circumstances and running a campaign for too long can cause factors to change, rendering your results void and inconsequential.
A/B Test Result Analysis
When analyzing your results, look to draw conclusions on the most actionable aspects of your marketing. Conversion metrics are the most important to keep in mind. Consider the user’s journey through the process: your ad appeared on their screen, they clicked on it and were taken to your landing page where they either completed an action or they didn’t.
Advanced A/B Testing
When you have gone through a few campaigns and start to understand how A/B testing works, you may be looking for more advanced techniques with which to elevate your campaign effectiveness. Here are some tactics that can boost the results you derive from an A/B test.
Testing Multiple Variables, Also Known As A/B/n Testing
The difference between testing too many variables and being able to successfully examine multiple unknowns has to do with the specificity of the experiment you design. A/B/n testing involves testing more than a single variant against your control, in this case, your initial landing page or ad. You can break you’re A/B test down to more than two versions, thus focusing on different aspects you want to test.
Instead of creating two ads that go to the same landing page, you can create four different ads that each have a specific, important variation. Conversely, you can send one ad to four different landing pages, each with a different conversion strategy.
Leveraging Remarketing
Remarketing is the ability to sell to a user that has visited your website but left without converting. Internet platforms like Google and Facebook provide you the opportunity to approach these users with your ads at a later time while they are on another internet location.
Here’s a simple graphic by Southern Web explaining remarketing:
Being able to include remarketing into your A/B test requires that you plan your campaign in a way that will allow you to cross-reference data from users that converted on their first visit and those that had to be marketed to on a remote destination.
Final Thoughts
Be thorough in your planning and conduct your A/B testing in a methodical fashion. You will enjoy significant results visible in your bottom line while simultaneously improving your visitors’ user experience and your brand’s online presence. Don’t be surprised if your newfound A/B testing prowess transfers to other areas of your business, such as research and development, human resources and logistics.
