A/B Split Testing with AdWords
Posted: February 1, 2005
There has been a lot of talk over the months about split testing ads & landing pages. Google allows this to happen pretty easily as they allow multiple ads to show in one group.
This is the setup for a very simple test to start people thinking about how to split test and measure such test results. There are, of course, much more complicated techniques to use. Most webmasters/marketers don’t have incredibly sophisticated tools, so this example is based on a low tech variation that can be run with just an excel spreadsheet.
If you think about an ad, you have 4 lines to work with. (Some consider it 3, depends on how you feel about the AdWords Display URL. Each line is both it’s own entity, and related to the whole ad - all at the same time. Each line must carry it’s own weight, and reinforce the message you’re trying to send via your ad.
Note: This test is for using 3 lines. 1 title, description line 1 & 2. If you like writing sentences (often useful if you’re in the premium position), then you’ll be testing the ad copy slightly differently depending if the sentence can be broken into halves and split tested using the below.
This is assuming we’re only testing ads, and not landing pages. It’s also assuming there is not a tag/branding line we want each ad to have (although, if you’re still trying to figure out how to brand your site and looking at tag lines for every ad - this is a technique that can help you find a line that ’speaks’ to people in your market). I’m going to ignore the display URL right now, as you should get the idea from the rest.
Turn off ‘ad optimization’ in the campaign settings, we want all the ads to receive equal exposure.
I would suggest ignoring dynamic insertion on the first trial run. This can be added later after you determine what type of copy works best for you. But for dynamic insertion to work properly, you also need to know what the ‘backup’ text needs to be as it will be shown sometimes.
Determine how many ads you can successfully test based on your volume (if you’re getting 10k impressions a day, you can test many more ads over the course of a 2 week trial than if you only get 100 impressions). At some point, you’re going to want to limit the total number of ads you’re running purely from an unwieldy amount of data standpoint.
Create 2-4 titles (T below), description line 1s (D1 below), description line 2s (D2 below), and then mix and match them. Remember, these go up exponentially, testing 3 lines is 27 ads (i.e. #t x #d1 x #d2).
Your ads will look something like this:
ad# T D1 D2
1. 1 1 1
2. 1 1 2
3. 1 2 1
4. 1 2 2
5. 2 1 1
6. 2 1 2
7. 2 2 1
8. 2 2 2
Using just 2 variations of each line, we’ve already created 8 ads, you can tell this number goes up quite quickly, a test of 4 different copies can quickly become a huge amount of work.
After you’ve hit your time frame for testing, put all your data into an excel spreadsheet. (I prefer a time frame of at least one week so I can receive visitors on every day of the week to test out the results. Often, I go for 2-3 weeks. If there is a holiday or special event in your week, then you should go another week as holidays lead to a different type of traffic. Some prefer to test by total number of impressions - it’s all preference).
If you’re using the Google conversion tracker, select ‘custom report’. Deselect ‘keywords’ as you don’t want to see every keyword. Select the AdGroup you want to test (you can also select several if you want to test how different lines work across your entire account - useful for testing ‘tag lines’) Select CTR, conversions, conversion rate, transactions, transaction rate, ad title, ad description line 1, ad description line 2.
Run reports by CTR, conversion %, total conversions, total transactions, transaction rate and by total profits (one ad could have a lower CTR but higher conversion %, thus produce a better bottom line (Forget ROI, Just give me profits). To get the number for total profits, you’ll probably have to use information from your intranet/shopping cart or use the advanced conversion tracking options and use your purchase variable in the tracking code.
Transaction rate is a useful number to use. It shows how many people were either repeat customers, or came back after visiting your page and then converted (conversions are for people who visited and purchased without leaving your site, transactions are the total number of times your page with the Google tracker & a cookie on their machine saw the "thank you" page). If you see ads with much higher transactions than conversions, you might want to find ways of converting them more quickly before they leave your site.
You’re going to download this report from Google using the CSV for excel link.
First, run these reports by all ads. This will give you a benchmark to look at for the average conversion % / bottom line of the aggregate numbers.
Then you’ll run them by title, description line 1, description line 2. In the above, you’ll have 12 reports (title 1, title 2, description line 1A, description line 1B, d2A, d2B, ads 1-6). Analyze these numbers carefully. Don’t just see which lines received the best ROI, profits, and CTR - look to see if one description seemed to work better with another description line or title line, etc.
In excel, a starting point to run such reports is to use the autosort feature. From the dropdown box choose to only show the line you want to measure. The autosum feature will then let you see exactly what that line received. For easy of use, copy that line into a different sheet along with what title/description line you were measuring(remember to paste variables only, not formulas). This will store all this data in one place for ease of analysis.
At this point in time, you should know exactly what lines gave you the best conversions, best CTR, highest bottom profit line, which are the second best, etc.
Optional step: Take the best performing ads (highest CTR, highest ROI, highest profits, highest conversion % - odds are, you won’t have 4 different ads, one ad will be the best for more than one), and if applicable, apply dynamic insertion to the title (for some ads, you’ll also be able to test dynamic insertion on lines 2 and lines 3).
Repeat the above as often as you desire. Of course, you’re going to always take the best performing ad or 3 and keep using them in the split test for benchmarking purposes.
If you want to split test landing pages, this is a good time to start.
First, remember:
1. Ads and landing pages have a relationship to each other.
2. Different ad & page combinations can lead to different conversion %s.
3. It’s possible your best ad and best page actually have a lower conversion %, but your second best ad and best page (or vice versa) produce better results.
4. Don’t assume you know what is best or what should work, let the numbers tell you the story.
If you want to test pages at the same time as ads, then once you make your ads, duplicate them exactly except for the landing page. (i.e. In the above example, you’ll have 8 ads to one landing page, and 8 identical ads leading to a different landing page) for a total of 16 ads testing two landing pages. Again, these are exponential numbers. If you’re dealing with low volume, choosing your top 3 ads and testing them with 2 landing pages might be better as you only need enough information for 6 total combinations (3 ads to landing page 1, 3 ads to landing page 2) and not 16.
When you begin to analyze the results, you are going to add another step in the reporting, landing pages. You’ll look at the overall conversion % of a landing page as a benchmark, and then compare how each ad also interacted with that landing page.
The results of such split testing can be very eye opening. At some point in time in your split testing, try making extremely similar ads with just a one word difference (a plural and a singular; two different ‘offer’ keywords; two different adjectives; etc.) At this point in time, you’re refining your ad copy and offers as much as split testing to learn more about your audience.
Split testing ads and landing pages creates additional work. This is very worthwhile time spent to get the most from your advertising dollars. Having access to toolsets can definitely speed up the process, although, it’s very possible todo all the required analysis with free Google tools and just a spreadsheet.
Hope this begins sparking the thinking process behind moving from striving from just high CTR ads to finding the best converting/profit margin ads.
Related Information:
- Some Ideas for Split Testing AdWords
- Google AdWords Split Testing Login Pages?
- Profit by Impression : The real metric in PPC testing
- AdWords Hypothetical Formula
- Google Testing Click to Call
Comments
8 Responses to “A/B Split Testing with AdWords”
Google AdWords
Seminars for Success
Learn about Google AdWords from experts hand selected by Google.
These seminars will educate advertisers on the creation and management of successful AdWords campaigns.
Upcoming Seminars:
Las Vegas Seminar Monday, November 17th
Washington DC area Seminar Tuesday, December 2nd
Houston Seminar Thursday, December 4th
Learn More about the Seminars:
Seminar Information
Official Google Seminar Page
Suggest a new city
Brad Geddes
Leslie Clark
Other Memberships
[...] You may find more about creation and variations of ads for split testing at this blog by Brad Geddes. March 21st 2006 Posted to Adwords Blog, Adwords tips [...]
[...] It is often useful in split testing or determining Profit by Click to know which ad was clicked for tracking sales or under preforming traffic back to the actual ad. This dynamic parameter adds the ‘ad id’ to the URL: {creative} [...]
[...] I was putting together some information about split testing landing pages, a post in response to A/B Google Ad Split Testing. However, it seemed someone decided to post some personal information about a Googler that prompted AWA to email me asking for some help in getting the mess cleaned up. [...]
[...] finally getting a chance to talk in depth about split testing, profit by impression, lifetime visitor values, and quality score to audiences who are hungry for [...]
[...] Don’t just assume you understand the best ad copy or landing page for a visitor. Constantly split test ads and landing pages to ensure you are making the best decisions about user engagement [...]
[...] a comprehensive description on how to test A/B testing of Text Ads, Landing Pages with [...]
[...] this post will already know the answer to this question, but it never hurts to have a reminder. Testing ad copy will allow you to incrementally experiment with variations in your messaging to determine which ad [...]
[...] idea is to split test your pages where the major change is layout based upon different screen resolutions before you do any major [...]