Perhaps the most powerful thing about paid search is that PPC campaigns offer instant feedback. This is what makes A/B testing so powerful: you’re able to funnel the firehose of data that pay-per-click campaigns fire at you into either of two theses and get instant feedback.
This is why a new AdWords feature called AdWords Campaign Experiments (or ACE) is so powerful. Basically this feature allows you to isolate certain aspects of your Google AdWords campaigns and test certain elements, splitting off traffic in whatever way you like. In this post I’ll walk you through:
According to Google you can test “keywords, bids, and placements.” So what does that mean, exactly? Joe Kershbaum of Clix Marketing had a great piece on five ideas you can try out with a lot of great information on things you can test, what you might do with those controls, and some of the things you can’t test — definitely take a look at Joe’s post for more information, but I’ll lay out a brief summary of what he talks about here:
Now that you have an idea of what you can test, let’s walk through setting up an experiment.
A great question you always get whenever you talk about testing is “what should I test?” Unfortunately the real art and opportunity of “testing” lies outside the mechanics of using tools like Website Optimizer and Campaign Experiments: you need to have good ideas about things that will likely improve your campaigns to create good test design. In the case of ACE, your testing impetus can be very data-driven:
There are many other examples of great things to test, particularly as you get into bidding experiments, but hopefully this gives you a general feel for the types of reactions you can make to your data and the types of experiments you might want to set up (if you have a great action/reaction example for an awesome split test, drop it in the comments!).
So we have our design idea: we want to take a campaign that’s been using broad match, and see what the impact would be when measured against a campaign that used all modified broad match. For this example we’ll do this simple transition, but you could just as easily test broad vs. phrase or broad vs. all your keywords on all match types.
First, you need to navigate to the campaign > settings tab and scroll to the bottom of the tab, under the advanced section:
Next you simply name your campaign, and determine the percentage of impressions you’re willing to dedicate to the experiment (if the campaign you’re experimenting with is core to your business, you may not want to jeopardize 50 percent of the impressions on the experiment in case results are worse, so you can choose to only run the experiment against say 10 percent of impressions and mitigate some of your risk):
Next, once you’ve saved the settings on this screen, Google will allow you to “start your experiment” — but that won’t work because we haven’t changed anything! Google doesn’t really hold your hand for this stage, but luckily we’re here to do just that.
Before running the experiment, you need to jump back to the Ad Groups tab and start to make your changes.
Let’s walk through converting an Ad Group from broad to modified broad. First, select the Ad Group you want to change within the campaign where you’ve enabled experiments, and toggle to the keyword tab. You’ll see an experiment icon next to each keyword:
Here we can determine what to do with certain keywords. In this case, we want to leave all of the existing keywords as the control, and set up a fresh list of keywords on Modified Broad match for our experiment. We can do this by selecting all the keywords via the check box next to the keyword header, and changing the status of all of our keywords to “control only”:
Now we’ve designated all of the broad matched keywords as our control. Next we need to generate our list of modified broad match keywords. For this task there are two awesome (and free) tools.
First, we can just export the existing keyword list from the AdWords interface by:
(Brad Geddes had an awesome video that will be a huge help in walking you through reporting in the new AdWords interface if you’re not confident with the new reports available).
Once you have your list of keywords, you can simply drop them into a free tool:
Next, grab the output from whichever tool you use and input your new keyword list into the Ad Group as experiment only keywords:
Click save, and you’ve set up the experiment! Now you just need to set it live and analyze results.
To enable the new campaign experiment you’ve created, you’ll need to jump back to the campaign > settings tab and scroll all the way to the bottom again. This time, you want to apply launch changes, and start running the experiment.
The final step, of course, is actually monitoring your results! For this, you’ll need to do a bit of data manipulation in Excel to really get a good picture.
First, to get the data, look at whatever aspect of your campaign you want to do the analysis against (the Ad Group or Campaign level to get a quick picture of performance, or at the keyword level to get a more granular view) and use the segment tab to create a report segmented by your experiment data:
The output will look something like this (though hopefully your results aren’t as blurry):
Note: you may have data labeled “outside experiment” — typically this is just the clicks and impressions that occurred before or after you ran your experiment.
Finally, we can take this report, download it into Excel, and start to get some interesting insights, such as:
From here we can see which areas of our campaign work better with modified broad, which areas sacrifice volume and/or cost per acquisition with the new match type, and react accordingly.
AdWords campaign experiments is a pretty awesome tool — if you have any anecdotes, use cases, things that drive you nuts about it, or specific tests you like to run with it leave them in the comments!
Tom Bates on the Epiphany solutions blog had a nice post on ACE as well f you’re really looking to dig in further with campaign experiments, here are a bunch of handy links from Google themselves:
Please read our Comment Policy before commenting.