AdWords Tips

Google Releases New Quality Score Info: What's Really New Here?

By Larry Kim June 26, 2014 Posted In: AdWords Tips Comments: 13

Google this week has published a new white paper on Quality Score and finally updated their 5-year-old video on how Quality Score works. 

I asked them “why now” and they said that it’s just time to redo the old stuff, but part of me thinks that they’re trying to quell a Quality Score rebellion that myself and others (including Frederick Vallaeys, etc.) may have inadvertently started – the wording of some points in the new materials seem to directly reflect points that I’ve articulated in articles I’ve published in the last year.

quality score

Image via Kirk Williams - @ppckirk

As usual, Google trotted out the same old “happy users,” “happy advertisers” and “happy Google” platitudes. It’s mostly a rehash of the same old Google fairy dust about Quality Score, but there are a few interesting nuggets in there worth responding to (and debunking).

Downplaying the Significance of Quality Score

Google says:

Quality Score Is a Helpful Diagnostic Tool, Not a Key Performance Indicator

Why: Your Quality Score is like a warning light in a car’s engine that shows how healthy your ads and keywords are. It’s not meant to be a detailed metric that should be the focus of account management.

I’ve often argued that Quality Score (or, essentially, Click Through Rate) is *the* most important key performance indicator to track in PPC, since it plays a huge role in AdRank which in turn directly impacts CPC, Ad Position, and Impression Share, and thus directly impacts number of conversions and cost per conversion. Across the many accounts we manage here at WordStream, the ones with higher Quality Scores are almost always better off than those with lower average Quality Scores. So why the heck are they downplaying the significance of the metric?

I think it’s because by definition, half of us have low Quality Score accounts (they grade on a curve – we can’t all beat the average expected CTR – by definition, half of us will be “below average”). By my estimation, 66% of Google revenues come from below average Quality Score keywords (due to the CPC penalties for low QS keywords and CPC discounts on high QS keywords). So I think it’s smart and understandable for Google to downplay the significance of QS – but you, as an individual advertiser, should know better.

check quality score

Furthermore, if the check engine light in your dashboard is flashing, it means your car could break down soon and you are endangering the lives of yourself and your passengers. Strongly disagree with Google on this one and if your “check Quality Score” light is on, I think you should definitely focus on fixing it.

Google Finally Admits Performance of Related Keywords Matters

This was pretty big – the new Google white paper says:

“For Newly-Launched Keywords, Performance on Related Keywords: Does Matter”

Note that Google has never explicitly stated this before. Now on this one, I’m pretty sure I’m the reason why they changed their stance. I recently pointed out that all accounts have keywords with no clicks and no impression data that still have Quality Scores, and that the “default Quality Scores” were always high for great accounts, and always low for terrible accounts. So clearly related keywords can affect the Quality Score of other keywords in your account.

The corollary of this is that it confirms another theory of mine – having keywords with higher average CTR/Quality Score has a beneficial impact on other keywords in your account, which is why I always run a branded keyword campaign. (Branded keywords get super-high CTR’s and thus can float your whole account higher. In other words, if you’re starting off with a high QS account, your new keywords will always have a higher Quality Score out of the gate.)

Alternatively, it also means you should delete the terrible low QS/CTR keywords that are killing your account so that the rest of the account can breathe.

Google Reveals the Weighting Factor of Ad Format Impact on Ad Rank (and is Likely an Error)

A few months ago Google announced that the use of Ad Extensions would impact AdRank but didn’t provide much detail into the precise weighting of the new “Ad Format Impact” factor.

In the new video, Hal Varian gives us 7 clear equations to work with, and using a little algebra you can reverse engineer the weighting of Ad Format Impact on Ad Rank in relation to other factors (Bid and Quality Score).

Four equations come from a calculation of ad rank @ 4:55 in the video:

quality score videos

And three more equations come later when he calculates their appropriate actual CPC based on the 2nd price of the auction at 6:15. (So the actual CPC would be the required bid to have an ad rank of the advertiser immediately below them – the first advertiser would need a bid of $1.73 to have an ad rank of 15).

quality score info

He doesn’t give us exact numbers to work with for Quality and Format impacts, so I’ll use discrete variables to represent the impacts of “High”, “Medium”, “Low”, and “No” rated influences of Quality (Q) and format (F).

Expressed as formulas, these 7 equations look like this, with ad rank on the right of the equation:

Let’s normalize all bids to $1 and directly compare ad rank across identical bids:

google quality score equation

What does this show us? Well, these numbers imply that the MAJORITY of ad rank is influenced by the impact of ad formats. For instance, take these 2 equations:

calculate quality score

Same quality impact, but the difference from moving from a “high” impact of ad formats from “low” doubles the ad rank. Some manipulation brings us to:

adwords qs equations

Implying that the impact of ad formats on ad rank is greater than the impact from Quality Score, which seems a bit over-stated in my opinion.

We suspect that Varian’s script wasn’t checked by his engineers as we see discrepancies in the numbers between minutes 5 and 6 once we normalize them for their bids.

For instance, consider the advertiser with $2 bids. His ad ranks don’t balance when you normalize:

google quality score white paper

The advertiser with $3 bids:

quality score hal varian

And the one with $1 bids:

quality score equations

It’s odd that he gives very specific numbers for bids that don’t translate correctly to ad rank, but it’s probably more a marketing video than anything else. Google gets lucky with how few people actually sniff these numbers for accuracy.

quality score video

Another reason I’m skeptical of Google’s example is that it doesn’t match up with our own customer data – for example you can look in your own AdWords accounts and see how CTR varies for ads with and without extensions – we did that a while ago and noticed that the use of ad extensions does indeed raise Click-Through Rates, as shown here:

ctr by ad position

And those ads with extensions raise Quality Scores too, as shown here:

quality score with sitelinks

In both cases you can see that there is indeed some uplift, but that it is modest and nowhere near as massive as the Hal Varian examples would have you believe.

For now, I suspect there’s a bug in the example and I’d hope that Google would correct it. (Thanks to Mark Irvine, our resident data scientist, for help with the equations here.)

User Device is Taken Into Consideration when Computing Quality Score

This is 100% true and validates our own internal findings.

new quality score

When looking at our customer accounts, we found that the average Quality Scores were similar regardless of the mobile share of account clicks, even though we noticed the expected CTR for mobile was very different from expected CTR for desktop. So the key takeaway is that Google uses lower expected CTR numbers when calculating QS from mobile.

They’re Still Pretty Vague on How Quality Score is Calculated

Google says:

Pay Attention to the “Big Three” Component Parts of Ads Quality: ad relevance, expected CTR and landing page experience.

This is true but they’re listing them out in a table as though they’re all equally weighted factors. Just beating expected CTR trumps all other factors by far. They should make this more clear. The old QS video had a pie chart showing the components of quality score where 2/3ds of the algo was based on CTR – I thought that was a better way to explain it.

Summary

They’re not wrong, but they aren’t telling you the whole story either for obvious reasons. The level of detail in the new white paper is higher than previously, which indicates that they were leaving out key information. I can only assume that the new video and white paper also omit key details. So when it comes to Quality Score (still the most important metric in your account), I think you’d be better off doing your own homework, as I have done, than taking the Google advice verbatim.

AdWords Performance Grader




If you enjoyed this post, please consider leaving a comment.

Comments

Thursday June 26, 2014

Terry Whalen (not verified) Said:

Hi Larry,

I think Google and Hal have noticed a lot of missinformation on quality scores, so they figured they'd try to elucidate on quality score a bit more - this recent post is nothing more than that.

You mention that you've often argued that quality score is the most important key performance indicator in AdWords, but since QS does not include conversions, revenue,or costs, it seems silly to think of it as a true KPI, no?

It's easy to increase quality score - just write more click-friendly ads that users will click on, without regard to truth-in-advertising or any ad-level user qualification. CTRs and quality score will increase, and CPCs relative to ad position will decrease, but this will likely lead to lower ROI. There is oftentimes an inverse relationship between conversion rates and ads that product high CTRs and quality scores. The AdWords manager should test for ads that generate the most conversions per thousand impressions, at the lowest cost per conversion. The winning ad may not be the ad with the highest CTR or highest "relevance" as measured by the AdWords QS.

It's a good thing to pay attention to quality score, since it's a proxy for relevance and user experience - but it's not something to obsess over.  That's the gist of what Hal Varian is trying to say.

 

Thursday June 26, 2014

Larry Kim Said:

I think the new info from the Googs validates the research that I and others have been working on. Would love to see any data to support your theory of the relationship between click through rates vs. conversion rates. We have previously published research on that topic and found no such relationship, nor has Google for that matter.

 

Thursday June 26, 2014

Terry Whalen (not verified) Said:

The article makes reference to this on page 8, middle of the page: 

As we mentioned about ad relevance, there may be times when a more specific ad leads 
to lower CTRs but higher conversion rates. Don’t assume one is always better than the 
other: look for the balance that leads to the best possible performance for your account.
 
They are saying the same thing - that there can be an inverse relationship between CTRs and conversion rates. This is quite common. If an ad is more click-friendly, it invites clicks from users at the margin - users that may convert at a lower rate than other users.

Thursday June 26, 2014

Larry Kim Said:

i think what they're saying is true here. YES it's certainly possible for lower CTR ads to produce higher conversion rates, so it makes sense to check. But it's equally likely that it will produce lower conversion rates. And on average, as you look at lots of accounts, you'll find that there isn't any inherent relationship between CTR and conversion rate. And given that is the case, it would make sense to get cheaper clicks (via higher CTR/QS) rather than more expensive clicks, all else considered equally {keyword selection, targeting, etc.}  - I'm talking averages here, in case that wasn't clear.

Put another way, my point is not that conversion rate isn’t important (it obviously is important), but that you should aim to optimize all of the components that go into Cost Per Conversion, including both CPC and Conversion Rate.

Thursday June 26, 2014

Terry Whalen (not verified) Said:

Well, you know, now we're coming closer to agreement! :)

Friday June 27, 2014

amit (not verified) Said:

Very well explained. Thanks a Ton for taking out time and sharing this in detail. Do you have a facebook / twitter fanpage??

Friday June 27, 2014

andres (not verified) Said:

Hi Larry....

Could you explain me something?? i have learned here it ( old Post )

Ad Rank= Max Bid * Quality Score

Your Price = [  The Ad Rank Person Below You / Your QS ] + 0,01

Larry....This formula doesn't works anymore????

Thanks a lot.

Andres M.

Friday June 27, 2014

Antoine (not verified) Said:

You shouldn't search something like if Quality score as performance score or just warning. Despite all its problems, Google just want to be pdagogic, like people don't think about paying less and more about attracting internet users. Quality score is used, but you can't spam or you shouldn't focuse yourself only on quality score but alos to any strategy. Google also wants to give the best results to its internet users, then they are using some relevant and creative strategies to evaluate ads. Just think you should be first on every technique and ttyyou should make all the stuff to get the lowest cost per conversion. It's the main message of Google

Sunday June 29, 2014

Wohllab Promo (not verified) Said:

Great Post…I like “Procrastination is a choice”

Sunday June 29, 2014

Markéta (not verified) Said:

Hi! Thank you for sharing your point of view. I have to admit that I do not agree at all.

I consider QS just a rough indicator, but I dont chase 10s. Why? Because qs is always calculated just for exact match kws. This is an important fact that makes qs just a helpful tool, but not a metric that sb should follow blindly. You are right that better qs = better results, but it can be applied on account/campaign/group level, not kw level.

The power of the quality on performance of other kws is not a complete surpise, because google stated before that the quality of the entire account matters. So actually no news :)

Monday June 30, 2014

AnonyNothing here mous (not verified) Said:

GREAT BLOG POST, insightful. thank you for the effort.

"the first advertiser would need a bid of $1.73 to have an ad rank of 15)."

WHY THIS

1500 with 1.73

AND NOT THIS

1.73  with 2000

 

 

Monday August 04, 2014

R. Dorsey (not verified) Said:

Hello Larry,

Great post!  I for one am glad to see Google at least attempting to be more transparent about its calculations of Ad Rank.  I am sure that they are leaving out key metrics, so in the end, any formula with missing variables can only be an educated guess at best (unfortunate for us in the SEM world!).  I appreciate your take on their explanation and I largely agree with your conclusions.

Here at SimplePart, we do not use Google conversion tracking as we provide ourselves with an entirely in-house conversion tracking system, so as far as Google is concerned, Ad Words ends with the click.  I have found that in our case, QS is not a consistent factor with ad price, position and cannot be taken as a KPI in our vertical.  Some accounts have higher QS and pay more for the same key words than others, with similar landing pages and ad copy.  It's just not consistent, because there are a dozen other (unknown) variables involved.

My conclusion is that, at best, QS should be improved when possible, but ultimately the most important factor is accuracy of advertising (to avoid unwanted clicks) and cost-per-click. Some of our most successful accounts have only 7 or 8 QS where as some of our other less successful accounts have 9s and 10s for the same words.  It's just one of many factors.

 

Sincerely,

 

 

Monday October 13, 2014

Quality Score (not verified) Said:

HELLO

LARRY

Can you help me to to brink up  the quality score ,on my campaign.

Regards:Ivan

Leave a Comment

Type the characters you see in this picture. (verify using audio)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.
 
Free Keyword Tool

Get thousands of relevent keyword suggestions - more,
faster, free!

Free Keyword Niche Finder

Discover profitable pockets of keywords for your
business.

Free Negative Keyword Suggestion Tool

Identify wasted spend before it happens and increase
your paid search ROI.

Contact Us | Company | Support | Site Map | Trademarks | Privacy Policy © 2007-2014 WordStream, Inc. All rights reserved.