Split Testing

Have you ever wondered whether your company should use the phrase “Learn More!” or the phrase “Try it Now!”? In order to find the answer to this question, split testing, also known as “coin flipping,” would need to be done. For example, assume your company creates two ads that are identical to each other, except that one ad says “Learn More!” and the other ad says “Try it Now!” Now that you have two (or more) ads that only have one difference between them, you start showing them, alternating which ad is shown.

Split Testing

In order to be statistically significant, these two ads must be shown enough times so that each ad has been shown to a sample of people that accurately represents the general population (or your targeted audience, at least). This means that ad 1 and ad 2 have been shown to approximately the same number and type of people, so they STILL only have one difference between them. Once enough data is collected, you can then compare click-through-rate and conversion-rate for each ad. Given that there is only one difference between the two ads, we can now assume that any differences in the click-through-rate or conversion-rate can be attributed to that one difference. So, for example, if we see that the ad saying “Learn More!” gets clicked 5% of the time and converts into a sale 3% of the time, but the “Try it Now!” ad gets clicked 8% of the time and converts into a sale 6% of the time, we would then know that we should include “Try it Now!” in our ads, instead of “Learn More!”. The keys to this testing are keeping the audiences as close to identical as possible and ensuring that enough data is collected for significance.

AdWords

In the past, split testing for ads (or anything) would have been much more time consuming and expensive than it is today. Fortunately, today we have Google AdWords, a relatively inexpensive, easy, and fast way to split test a wide variety of things. AdWords is most commonly used to promote e-commerce sales. However, author Ian Ayres showed how AdWords can be used in other ways. Ayres published the book Super Crunchers in 2008. It is truly a book for data nerds, giving real life examples and explaining how analyzing data can solve a wide variety of problems. Within the book, Ayres explains how he settled on the title Super Crunchers for his book. He used Google AdWords. Ayres created two identical ads to promote his book, yet they had different titles. The ad with Super Crunching as the title received a significantly more traffic, thus assuring him that it was the title he should use. Google AdWords was a quick and easy method for choosing the title which would likely generate better sales. In the past, marketing research experts would have been hired at overwhelming costs. But, now we can use AdWords to answer some of the same questions, whether it is for e-commerce or the title of a book, at a faster and more cost-friendly rate.

What to Split Test

Now that we know AdWords can be used to split test a variety of things, we must decide what exactly we want to split test. Considering that FSC Interactive is an agency specializing in online marketing, we would want to split test ads that drive traffic to the websites of our clients. This may sound like a general statement, but there are actually a number of things within a single ad that can be tested. This infographic from Search Engine Land gives some great suggestions on what to split test. An example is testing whether to use all the space in the ad, or use a little of the space in the ad. You could test using an extended headline versus not using an extended headline. The results for these tests may vary among different industries and target audiences, so there are no set rules on what is the best practice. The only way to know for sure what your company should do is by split testing everything imaginable, over and over again. Luckily with AdWords, this is fairly easy, quick, and cheap to do.

Testing Guidelines

When split testing ads in AdWords, make sure that your settings are set to rotate the ads evenly. This ensures that each ad gets the same amount of exposure and is shown to similar audiences. Also, it is definitely possible to test more than two ideas against each other, and to run multiple tests at the same time. This allows you to gain more information at a faster pace. Make sure, however, that you collect enough data to make the correct decision. If each ad is only shown 10 times, that is probably not enough impressions to ensure that the data represents the reactions of the general audience on average. Lastly, AdWords Editor can be a great tool for split testing, as it makes managing ads in bulk easy. If you have a large, complex AdWords account with many campaigns, many ad groups, and many keywords, AdWords Editor will allow you to manage these with more ease. If all of this sounds like a different language, or too laborious and overwhelming, then please contact FSC Interactive and allow us to do all the hard work for you!