AdWords Tips

The Change in Quality Score Reporting: What Really Happened?

By Andy Stefano August 26, 2013 Posted In: AdWords Tips Comments: 25

A few weeks ago, Google announced that it was changing the way Quality Scores would be reported. Not how they are calculated, they pointed out, just how the scores are reported. This seems like clear confirmation from Google that the 1-10 Quality Score you see in AdWords is just a “front” for whatever your “real” score is.

If nothing changed about your “real” Quality Score, and this is truly just a glossy reporting change, then there should be no effect on your other metrics. BUT, there’s another way to interpret Google’s announcement. Google might be saying that they haven’t changed the factors that go into Quality Score, but they have changed how they normalize the results (which is how they can compare disparate advertisers). If that’s true, then there could be substantial impacts to your account. Quality Score is half the equation to determine your Ad Rank and an important part of determining the price you pay.

Quality Score Changes

Either way, if you carefully track your Quality Scores (or ask your agency to do it for you), you probably noticed a sudden change with no logical explanation. Since the announcement by Google, we’ve been watching our accounts carefully, trying to figure out just exactly what Google was planning to do.

The first challenge was looking for the date of the change. Google hadn’t told us exactly when it was coming, so we had to stay on top of our tracking and keep “slicing and dicing” the data. After all, we didn’t even know exactly what we were looking for. Was it going to affect all keywords? Just branded terms? Just low-quality score keywords? Generally raise scores? Generally lower them?

All in all, it actually took us until a few days after the fact to be sure of what we’d seen. And what we found was ...

Google Took Away Your 10’s

Quality Scores (the ones we can see) have generally fallen into a bell curve. For a long time, when we asked prospective WordStream clients “What do your Quality Scores look like today?” they all said “Mostly 7s.” More recently, they said “Mostly 4s.” But even in that period, most people had a healthy number of 10s that drove a good amount of their traffic – branded terms, by and large. Which makes sense. As Elisa pointed out last week, you are obviously the most relevant advertiser on your brand. Now, that’s simply not true. Some of our brand terms still have Quality Scores of 10. But a lot of them now have 7s and 8s.

Google Quality Score Reporting Changes

Without going into specifics, our clients’ accounts in the past have generally gotten about 20% of their impressions from keywords with a Quality Score of 4 and another 20% from their 10’s, with a  relatively smooth curve between. It ends up looking like a roller coaster. Since the change, however, we see something that looks more like a traditional bell curve, with about 20% of the impressions coming from 5’s and a smooth curving going down in each direction. (Except for that funny little blip tail at Quality Score 1. Isn’t that weird?)

What does all this mean?

A few things. First, please don’t punish your agency for your recent drop in Quality Score 10’s. It’s all Google’s fault. (And Google’s not punishing you, either.)

Second, mark this date in your calendar. If you’ve been measuring the impact of your optimizations on your Quality Scores, August 2 will be a “break” in your data. Similarly, if you are a huge quantitative analysis nut and are making optimizations based on a correlation between CTR, Average Position, and Quality Score, you’ll need to redo all that math.

Only time will tell whether this change in Quality Score is just a reporting change or something more significant. I would expect, over time, for average cost-per-click to go up if Google is no longer giving out the Quality-Score-10 discount as freely.

Caveats

I should close by saying that (hopefully obviously) I’m working with a small sample size. I looked at data from WordStream’s client base, which tends to consist of small and medium-sized businesses rather than huge enterprise accounts. And I tried to control for other changes that might have impacted Quality Score (basically looking for campaigns where “all else is equal”), but our clients are constantly optimizing their campaigns. All this is to say “YMMV” – your mileage may vary. In fact, one other marketer did find something very different (a shift to higher scores). However, Martin Roettgerding saw results closer to ours – fewer 10's, more 7's and 8's.

How about your account? Have you been tracking your Quality Score changes? Did you notice a loss of 10’s? Sound off in the comments.

Andy Stefano is a longtime WordStreamer who has worked on the marketing, sales, product and quality assurance teams. A former teacher and PPC manager, Andy also trains new WordStream employees, does data analysis and makes WordStream fun. Andy lives in the Boston area with his wife and children.

AdWords Performance Grader




If you enjoyed this post, please consider leaving a comment.

Comments

Monday August 26, 2013

Victor Pan (not verified) Said:

Hey Andy,

Were these changes just cosmetic? (e.g. Your 10 became a 7, but you're still paying the same CPC)

Thanks for digging in! 

Tuesday September 03, 2013

Andy Stefano Said:

I want to give it a little more time and maybe expand my research to some other accounts before I say for sure, but that's definitely the million-dollar question. If it's all cosmetic, then it hardly makes a difference. But if it's not and our CPC's are changing too...that's a horse of a different color.

Monday August 26, 2013

Adam (not verified) Said:

Hi Andy,

Thanks for digging into this data. I'm not surprised that Google made the change to a more "normal" bell curve. Hopeuflly this will make the distribution more intuitive and help people understand how their quality really measures up (in Google's eyes).

-Adam

 

 

Tuesday September 03, 2013

Andy Stefano Said:

I think it will make things easier. Having so many 7's and 10's, but practically no 8's or 9's always confused people. (I remember trying to explain it to senior leadership once!) Normal bell curves are so much simpler to talk about. It's too bad that in order to get to "simple" conversations later, first we have to go through the explanation of why we lost all our 10's.

Monday August 26, 2013

Richard H. (not verified) Said:

Thanks for the update, I didn't even hear about this. I wonder what the real reasoning was behind this.  Was it just a massive update on Adwords or something else altogether different?
 

 

Tuesday September 03, 2013

Andy Stefano Said:

Thanks for reading. Here's the link to the Google announcment: http://adwords.blogspot.com/2013/07/improving-quality-score-reporting.html

Tuesday August 27, 2013

BSc Computing (not verified) Said:

This is what i wanted to learn, found these Google recent quality score changes but was unable to understand. Many thanks to write detailed post.

Tuesday September 03, 2013

Andy Stefano Said:

I'm glad I could help!

Tuesday August 27, 2013

Dino Kestic (not verified) Said:

Hi Andy,

Thanks for presenting this.

I work with a base of clients similiar to yourself within a online marketing company in Sweden. I've also noticed this change in QS for my clients and their accounts. Keywords that were on the scale between 4-6 in QS dropped to 1! Some got a bump higher up on the scale though.

So I contacted Google Support but of course, they didn't give me a logical explanation. 

I wonder if this is just "for show", cosmetics like Victor wrote or if they actually changed they way they compare advertisers?

 

Tuesday September 03, 2013

Andy Stefano Said:

Keep an eye on your other metrics - average position is where I would look to see if there was a change in how they compare advertisers. Of course there could be confusion with lots of other factors (a new competitor joining or an old one dropping out, for instance). But if average position starts moving in unexpected ways, keep this in mind as a possible factor. I plan to double check with our clients as well!

Tuesday August 27, 2013

Tim (not verified) Said:

Thank you for this article. I did notice major changes with the two largest accounts I monitor. One had all the quality scores moved up to 4 or higher. The second had a large number of scores bumped down to 2's and 3's. I was wondering what happened!

Tuesday August 27, 2013

Elisa Gabbert Said:

Hi Tim! Hopefully performance won't be affected!

Wednesday August 28, 2013

Phil Pearce (not verified) Said:

Clearly the mean avg has now changed from a 7/10 to a 5/10 looking at the above graph.

My view... is that only the Tripwire penalties (Relevance, Landing page, IsMobileOptimised) which are blended with VisableQS have been updated.

Previously I believe these penalties were:

  • Above Average: +5% to VisableQS
  • Average: +2.5% to VisableQS
  • Below Average: 0% no change

e.g. an actual QS of 5/10 + 5%*3 for Above Average: Relevance, Landing page & MobileOptimised = 7/10

Now I think they are using:

  • Above Average: +0% no change
  • Average: -2.5% to VisableQS
  • Below Average: -5.0% to VisableQS

e.g. an actual QS of 5/10 + 0%*3 for Above Average: Relevance, Landing page & MobileOptimised = 5/10

This would explain why the "top of the bell" no longer exists, and why you see a mini-spike a 1/10 where the 3*-5.0% is triggering.

Your thoughts?

Thanks

Phil.

Ref: penalties in Adwords API:
https://developers.google.com/adwords/api/docs/reference/v201306/AdGroup...

Wednesday August 28, 2013

Christine Derrel (not verified) Said:

Phil, thanks a lot for that link to AdWords penalties!!! Learned a lot!!!

Wednesday August 28, 2013

Randall Magwood (not verified) Said:

Was never a big fan of quality score - i guess cause my were always at an acceptable level. But in some niches of mine, they were poor. With the changes, i'm definitely going to have to stay on top of my game. Great reporting.

Tuesday September 03, 2013

Andy Stefano Said:

Thanks for the compliment. I can understand not being a fan - it's a fuzzy metric that (clearly!) can change at Google's whim, rather than the sort of cold-hard metrics that are more in your control. But it's a great bit of insight into where you need to (and can) improve.

Thursday August 29, 2013

Fade Adenile (not verified) Said:

Thus is the life of Online marketing. Ever changing and ever adapting. Adapt or die I guess you could say.

Tuesday September 03, 2013

Andy Stefano Said:

That's what I like about it. (Well, not the dying part.) But it does keep you on your toes.

Thursday August 29, 2013

Stoixima (not verified) Said:

Hi

Your post is really interesting.

The google recent updates in the quality score was a little bit weird and your info is very helpful.

Regards from Greece

Tuesday September 03, 2013

Andy Stefano Said:

I thought it was weird, too! 

Wednesday October 16, 2013

Natalia (not verified) Said:

Great article! Could you please let me know where I can find the new reporting. I tried to add QS as a column and I still get the 'old' QS

Thanks!

Thursday January 02, 2014

Sahil (not verified) Said:

Really Useful info, Best part is that its very Clearly written which ensures good understanding for the users.

Tuesday March 04, 2014

pame stoixima (not verified) Said:

Hello Andy !

I'm a little bit confused about quality score by definition but i can start to understand by contextual meanings. I wanted to ask if there is any other info apart from http://adwords.blogspot.gr/2013/07/improving-quality-score-reporting.html, for the quality score basics, or other adwords meanings basics.

 

Thx

 

 

Monday June 09, 2014

Spook SEO (not verified) Said:

Hello Andy!

You got a great presentation here. I learned something new from your post. Anyway, I tend to agree with you that only time will tell whether these changes in Quality Score is just a reporting change or something more important that we should take into consideration.

Wednesday June 11, 2014

karen duff (not verified) Said:

Great article! Really Useful info

Leave a Comment

Type the characters you see in this picture. (verify using audio)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.
 
Free Keyword Tool

Get thousands of relevent keyword suggestions - more,
faster, free!

Free Keyword Niche Finder

Discover profitable pockets of keywords for your
business.

Free Negative Keyword Suggestion Tool

Identify wasted spend before it happens and increase
your paid search ROI.

Contact Us | Company | Support | Site Map | Trademarks | Privacy Policy © 2007-2014 WordStream, Inc. All rights reserved.