The figures in Google Webmaster Tools (GWT) search query report can provide some pretty nice looking metrics that many marketers would like to take advantage of. The problem is that they often come under scrutiny for their accuracy. Even Rand mentioned it during his AMA on the BigSEO subreddit.
Link to mentioned article: Why you should not use Google’s WMT data
I won’t talk about the accuracy of impressions, clicks, or CTR. Instead I was more focused on the accuracy of the Avg. position metrics. Like Rand said it uses averages, which can be real misleading when you do not know the range of positions for your site when it showed up in the SERPs.
In the past, I haven’t given much credence to the Avg. position in GWT’s search queries report for two reasons:
- It is an average over a period of time, and I/clients want to know where the website ranks now, not the ranking over the course of month.
- I have paid software dedicated for tracking keyword rankings, therefore I assume it to be more reliable, honest and up to date.
Despite my skepticism I decided to run a few test across a number of keywords (1,100 to be exact) for three separate web properties and see how GWT stacks up to the paid software.
What I Want to Get Out of This
There were a number of answers that I wanted to get out of this test:
- On average what was the difference in the rankings?
- What is the average absolute difference between the rankings?
- How many more keywords did GWT provide rankings for that the paid software missed?
- How many more keywords did the paid software provide rankings for that GWT missed?
Some Things to Consider
This test will only be comparing the keyword used in the paid software, with nearly all of them being geographic specific or branded.
- While all of the keywords are relevant to the clients, not all will be as heavily prioritized as others.
GWT will not show you any metrics unless the websites are receiving impressions for the keywords.
- This gives an advantage to the paid software to show the rankings for more of the targeted keyword, since you actually tell it which queries to check for regardless of impressions or average monthly searches.
We have set up the paid software to scan only the first 50 SERP 50 positions.
- This gives an advantage to GWT, since for some keywords it has showed an average position up to the 520th position. So it will be able to provide information for the keywords that are deeper in the SERPs that the paid software will not reach.
The keywords used are a mixture of keywords researched by us using tools with actual search metrics, keywords that are thought to be useful but show no data in keyword research tools, and some keywords requested by clients.
- This will most likely give an advantage to the paid software to check the rankings for keywords that may rarely ever get searched or not at all.
I will only be using the www. version of the profiles from GWT, and am not sure what keyword data will be missing by not including the non-www. version.
How the Test Was Run
The way I conducted the test was by using only the keywords (1,110) that we had being tracked in the paid software, since these are the keywords that are most meaningful to the client. I downloaded the data from GWT and the paid software, exported it into excel and used the VLookup function to combine and compare the data for the keywords. If you know how to export and use simple Excel functions, you can try this at home for yourself.
Results for Keywords Both Tools Reported On
Combined, GWT and the paid software provided ranking data for a total of 579 of the 1,100 (52.5%) keywords, and both provided ranking data for the same keyword 331 times (31.1%).
When both tools provided data for a keyword the Avg. position in GWT was -.69 positions lower than the positions in the paid tool. The tools had an average absolute difference of 1.92 positions per keyword.
The largest absolute difference for a keyword was 23, where GWT provided an average position of 23 and the paid software provided a ranking of 46. Only 7 out of the 331 (2.4%) keywords that both tools provided data for had an absolute difference greater than 10. 61 of the 331 (18.4%) keywords matched exactly between GWT and the paid software.
| Absolute Difference | Occurrences | Percentage |
| Greater than 10 | 7 | 2.11% |
| 10 through 3.1 | 37 | 11.18% |
| 3 through 1 | 178 | 53.78% |
| Less than 1 | 109 | 32.93% |
Providing Missing Data
GWT was able to provide the rankings for 113 keywords that the paid tool did not give, 68 of which were under the #50 position threshold of the paid software. The software may have missed some keywords that should have showed in the report. There could have been a glitch, or these keywords could have had few impressions with a wide range of positions, making Avg. position seem more attainable for the paid software. The lowest average position found for a keyword by GWT was 96.
The paid software provided data for 135 keywords that GWT did not have, meaning that these keywords did not have any impressions, according to GWT.
Conclusion
When both tools provided data for the same keyword, they compared much more closely then I would have assumed. If before I conducted the test, you would have told me that 86.71% of keywords in GWT and the paid tool would have an absolute difference of 3 or less, I would have shook my head. I expected the numbers to be all around the map and very inconsistent.
Considering that GWT provides you with an average, you have to assume that the more impressions that a keyword has, the more accurate their Avg. position is and vice versa. If hypothetically a keyword only has 2 impression and an Avg. Position of 10, it could have ranked 9th and 11th for those searches or 1st and 19th.
For the best results in keeping track of your rankings, it is best to compare your keyword ranking tools along with GWT. GWT was able to provide data that the paid software missed, and the data from the test seems to be trustworthy enough to assume that it is relatively correct.
However, the paid tools will be able to give you more control as to which keywords you get data for, regardless of the amount of searches and impressions. In the paid tools, you can make sure that the keywords are all relevant, while in GWT you will get a mix of data for relevant and irrelevant keywords that you have to sort through.
Clients want to see the progress that is being made and one way is to show them where their site shows up in the SERPs. They may have their own keywords in mind, which will bring them little traffic, but they still want to see that they are ranking, and this is not information that you can comfortably assume will be supplied by GWT.
GWT is still a great tool to find keywords not previously targeted that you are on the edge of getting to page one. Exporting and sorting the data is a great way to find those keywords and make the most out of the data.
If you do not need reports dedicated to the rankings for certain keywords, then GWT will most likely provide you with the data that is accurate enough to help you with the development of your online marketing strategy. Just remember the following:
- It is important to keep in mind that if you know you are ranking and it doesn’t show up in GWT, the keyword probably isn’t worth your time, due to no one searching for it.
- If you know the keyword is searched for but doesn’t show up in GWT, you will need to do a better job of optimizing to increase your rankings so that your site will be found.
The unknown question is which is more reliable? With the results being so close for so many keywords, which do you trust to be more accurate for those keywords that had the larger absolute differences? The main keywords that puzzled me were those that GWT picked up, but the paid software missed. It was understandable that GWT wouldn’t have data for them all since they may not have had any impressions. The paid software obviously wouldn’t have them all either, since it would only scan 50 positions deep. However, 68 of the keywords that the paid tool missed had an average position of 25.4, which means that it should have been able to record data for some of them.
This test was run with data from only two data providers, GWT and a paid tool. It would be interesting to run the test with a third or fourth party to see how the results differed. It would have also been useful to use more keywords to compare, preferably those that are broader and have more monthly searches. But for now, I am satisfied with the results, and this test has restored some faith in the data provided from Google Webmaster Tools.
