Case Study: How Much Do Rankings Matter?

We’ve heard a lot of speculation lately about the future of rankings – the SERPs are clearly evolving, and what was once a simple list of 10 results has gradually become personalized, localized, and wikified. It begs the question: As the SERPs diversify, do rankings matter as much as they used to? Recently, I had an opportunity to collect some data on this question. Earlier this year, a client suffered a ranking drop for their primary keyword (likely connected to Google’s alleged brand-related changes), which has recently recovered. So, I decided to run some numbers to see how that ranking drop and subsequent recovery affected search traffic.

1. Direct Keyword Traffic

The graph on the left (I) shows the client’s rankings, observed weekly, for their primary keyword over a period of about 7 months. The right-hand graph (II) shows weekly search traffic for that same keyword:

A couple of notes: (1) this is a one-word keyword, (2) it’s relatively common/popular, and (3) rankings over the 7 months fluctuated from 1st to 5th (I’ve inverted the graph to show 1st at the top). I’m a big believer in eyeballing data first, and I think the graphs show some degree of connection between ranking and direct traffic. The visual is a bit more compelling than the correlation (r = -0.31), although there is some relationship. The negative correlation is expected and reflects the fact that a low ranking is better (1 > 5, value-wise).

2. Long-tail Organic Traffic

From a broader SEO standpoint, though, how did the ranking of the client’s most important keyword affect their overall search traffic? I’ve repeated the ranking graph (I) below, but added a graph (III) of weekly traffic from Google organic results over the 7 months:

Here, the relationship seems to fall apart (r = 0.21). Other than a late-summer dip, overall organic traffic from Google actually grew as their primary keyword rankings were falling.

What Does It All Mean?

So, do rankings matter or don’t they? Well, in terms of direct traffic for the specific keyword that suffered the ranking drop, there’s certainly some effect. Studies have shown pretty reliably that search visitors focus (and click) most on the Top 3 results, with #1 getting the lion’s share. Outside of the single-keyword view, though, the situation gets a lot more complex. While the client’s primary keyword is an obvious choice for their industry and is fairly competitive, the traffic for that keyword accounted for only 1.3% of their total organic traffic from Google over the 7 months. If we had chosen to take a very narrow view, and obsessed over their primary keyword, we would have neglected over 98% of their SEO results.

Of course, I’m not saying rankings are irrelevant, just that you need to take a broader view, including:

• Tracking diverse, long-tail phrases
• Tracking search phrases actually used by visitors (not what you think they use)
• Measuring search analytics, such as total traffic from search

While the phrase “long tail” may feel like it’s been beaten to death, the impact of the long tail is becoming more relevant every day. Over the 7 months in this case study, Google visitors used over 250,000 unique phrases to reach the client’s site. Over 80% of those phrases didn’t contain any variation of the primary keyword at all. So, while rankings obviously still matter on a keyword-by-keyword basis, being #1 for your top keyword (or even your top few keywords) is no longer good enough – if that’s all you’re measuring, then you’re missing the big picture.