Angeles Cielitolind: Did Google's Site Diversity Update Live Up to its Promise?

Posted by Dr-Pete

On June 6th, on the heels of a core update, Google announced that a site diversity update was also rolling out. This update offered a unique opportunity, because site diversity is something we can directly measure. Did Google deliver on their promise, or was this announcement mostly a PR play?

There are a lot of ways to measure site diversity, and we're going to dive pretty deep into the data. If you can't wait, here's the short answer — while Google technically improved site diversity, the update was narrowly targeted and we had to dig to find evidence of improvement.

How did average diversity improve?

Using the 10,000-keyword MozCast set, we looked at the average diversity across page-one SERPs. Put simply, we measured how many unique sub-domains were represented within each results page. Since page one of Google can have less than ten organic results, this was expressed as a percentage — specifically, the ratio of unique sub-domains to total organic results on the page. Here's the 30-day graph (May 19 – June 17):

A site diversity of 90 percent on a 10-result SERP would mean that nine out of ten sub-domains were unique, with one repeat. It's hard to see, but between June 6th and 7th, average diversity did improve marginally, from 90.23 percent to 90.72 percent (a 0.49 percent improvement). If we zoom in quite a bit (10x) on the Y-axis, we can see the trend over time:

Zooming in to just a 10 percent range (85–95 percent diversity), you can see that most of the change happened in a single day, and the improvement has remained in place for the week since the update. Even zooming in, though, the improvement hardly seems impressive.

Was the improvement more isolated?

Being as fair to Google as possible, we need to consider one of their follow-up statements:

What if Google improved the worst-case scenarios, but it wasn't immediately clear when we averaged out all SERPs? We can isolate situations with more than two listings from the same site by looking specifically at SERPs with a site diversity score of 80 percent or better (eight out of ten sub-domains are unique). Here's the 30-day graph for just those cases:

On June 6th, 84.58 percent of sites in our data set had a diversity of 80 percent or better. On June 7th, that increased to 86.68 percent — a 2.1 percent improvement. Let's dig even deeper to see what's happening for individual counts.

How did the impact break down?

A single data point doesn't tell us much about what's happening within each of the buckets. For this analysis, I'm going to use the exact duplicate count, since percentages can get a bit confusing once we have to put them in bins. Another complication is that, on occasion, two sites have more than one organic result — this brings down the overall diversity of the SERP, but doesn't necessarily mean that one site is dominating.

So, what if we look just at the count of the dominant site (the site with the most duplicates) across the 10,000 SERPs? We'll compare June 6th (blue) to June 7th (purple):

For slightly over half of SERPs in our data set, there were no duplicates (every site had one listing), and this number didn't change much after the update. The number of sites with two listings (i.e. one duplicate) increased pretty noticeably after the update (up by 346 SERPs). This was offset almost entirely by a decrease in SERPs with three to five listings (down by 345 SERPs across the three bins).

The numbers get too small to see at 5K scale after the four-count SERPs, so I'll restrict the Y-axis:

SERPs with dominant sites owning six to ten organic listings accounted for only 117 of 10,000 SERPs (just over 1 percent) on June 6th. After the update, this actually went up a tiny bit, to 119 SERPs.

We still see SERPs where one site dominates, and that story didn't change much after the update. That said, these six to ten-count SERPs are fairly rare. Looking at the keywords, we also see that many of them have brand or navigational intent. Here are a few keywords where we still see a ten-count:

"kohl's hours"
"macy's hotel collection"
"lowes outlet"
"dillard's sales"
"edd unemployment"

Many dominant-intent searches show site-links in the #1 position (which allow up to six additional links from one site). It's hard to say why Google isn't using site-links in these extreme cases. These may be situations where the intent isn't quite as clear, but we can only speculate based on looking at a handful of examples. Keep in mind, too, that Google determines intent algorithmically, so it can shift over time.

This isn't an easy problem. Site diversity isn't a lever you can pull in isolation, especially when it's left to the algorithm. Reducing repetition too much could harm quality, in some cases (especially SERPs with brand intent). Similarly, many algorithm updates unrelated to diversity seem to have unintended consequences for site diversity.

So, what's the final verdict?

When evaluating site diversity, we have to be careful relying too much on anecdotes. Anecdotally, there are definitely SERPs where a single domain seems to have too much power. For example, here's the main results column on a search for "pure green coffee extract" (I've removed a local pack for the purposes of this post):

The shopping results at the top suggest commercial intent, but the organic results are a mix of informational and commercial results. Amazon has a block of five product results, and this is not a situation where the query suggests brand or navigational intent (I haven't indicated any specific interest in Amazon).

It's easy to cherry-pick, and we can certainly say that Google hasn't solved the problem, but what are the broader results telling us? It's fair to say that there was some amount of improvement, and the improvement tracked with Google's public statements. SERPs with three to five results (two to four duplicates) from the dominant site decreased a bit — in most cases, these SERPs still had two results from the dominant site (one duplicate).

Even isolating the change, though, it was fairly small, and there was no improvement for SERPs where six to ten results came from the dominant domain. This may be because many of those queries had strong brand or navigational intent, and the six to ten count SERPs were rare both before and after the update.

While the improvements were real and Google's statements were technically accurate, the impact of the site diversity update doesn't feel on par with a pre-announcement and the PR it received. Regarding the state of site diversity in SERPs, Google has made minor improvements but still has work to do.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!