For a long time, if you asked me about spamming the search engines, whether with hardcore black hat tactics or merely gray hat link acquisition, I’d say that in the long run, neither was the right move. Building a great site and a great brand through hard work, white hat links, solid content and marketing strategies has always been my path of choice. It still is today, but my faith is definitely wavering.
Why?
In the last 12 months, I’ve seen (or, at least, felt) less progress from Google’s webspam team than in any previous year I can remember. Popular paid link services that Google’s search quality folks are clearly aware of have worked for months on end (some have done so for years). Crummy, low quality directories and link exchanges have made a comeback since the big shutdowns in 2007-8. Even off-topic link exchanges, which experienced their own blowback in 2006-2007 have started working again. Horrifyingly bad sites are ranking atop the results using little more than exact match domain names and a few poor quality links. There’s even a return of the link farms of the early 2000s, with operators creating (or buying old domains and converting them into) junky, one-page sites to boost their own link popularity.
On nearly every commercially lucrative search results I pull up these days, I see bad links pushing bad sites into the top rankings at Google.
Examples of Web Spam in the Rankings
I made a promise to Aaron that I wouldn’t “out” spam, and although I still don’t believe it’s the wrong thing to do morally (it hurts everyone’s search/web experience, why should SEOs band together to protect it?), I do want to keep that promise. So, while I can’t point you to any particular links or sites, here’s a good set of queries where plenty of link manipulation is keeping a few, some or many of the top (5-10) ranking sites in those positions:
Just run a few OSE reports on some sites that rank well here and you’ll see what I mean. There are numerous players in these listings who don’t have a single natural or editorially endorsed link. And you don’t need to limit yourself to these queries either.
3 Steps to Find Lots of Link Manipulation
Step #1: Search for “SEO Friendly Directory” and visit a few of the sections included in the resulting sites that come up.
Step #2: Search for the primary keywords the directory-listed sites are targeting in their title tags or the anchor text they’ve gotten from the directories.
Step #3: Check out the top 5-10 listings in the rankings and you’ll find an abundance of sites with few to no “natural” links whatsoever
Why is Google Letting So Much Spam/Manipulation Go Unpenalized?
I don’t know. But, I do have some guesses:
- Scalability of Spam Fighting Tactics – it could be that the ability for Google’s team to combat web spam has diminished due to the increasing size, complexity and demand in search. Perhaps fighting spam is a much tougher problem in the 100s of billions of pages than it was in the 10s.
- They’re Working on Something Big – for many years, Google would let lots of spam they clearly knew about pass… for a while. Then, they’d release an algorithmic update to defeat a huge layer of spam or seriously cripple certain types of link manipulation. If that’s the case today, this would be one of the longest times between updates we’ve seen (MayDay had a small impact, but it wasn’t link-manipulation targeted from everything I’ve seen).
- Too Much Baby Thrown Out with the Bathwater – perhaps, as link manipulation and spam have grown in popularity, Google’s found that they can’t penalize a technique or sites employing it without dramatically reducing the usefulness of their index (because so many “good,” “relevant” sites/pages do some dirty stuff, too). If this is the case, they’ll need to work on much more subtle, targeted detection and elimination systems, and these might be substantially harder to employ.
- WebSpam Team Brain Drain – The spam fighting team put together by Matt Cutts from 2001-2006 was Google’s cream of the crop. He personally hand-selected engineers from search quality (and other departments) to combat the black hat menaces of Google’s early growth days. SEOs could frequently interact with many of these crazy smart folks, from Brian White to Aaron D’Souza to Evan Roseman and many more. That interaction today is largely limited to the webmaster tools team, which may be an appropriate PR move, but it’s hard to know whether the new team is up to the task. We do have one new, semi-publicly contributing webspam team member, Moultano, on Hacker News (you can see all the threads he/she has participated in on the spam topic with this query).
Matt himself is finally taking a well deserved break, but even at home he’s much less public on the web, much less active on webspam topics on his blog, visits fewer conferences and now invests in startups, too (which surely takes up time). I don’t mean to criticize Matt in any way – if I were him, I’d have left Google long ago (and he’s clearly put in more than his dues), but the possibility remains that the team he built is no longer intact, or no longer of the quality it was in the early years.
- Live and Let Live – It could be that although Google’s public messaging about webspam and link manipulation hasn’t changed, internally their attitude has. Perhaps they’ve found that sites/pages that buy links or run low quality link farms aren’t much worse than those who don’t and having relevant results, even if they’ve used black/gray hat tactics, isn’t highly detrimental to search quality. Certainly in some of the examples above, that’s the case, while in others it’s less true. I recall that years ago, the MSN Search team noted that they’d much rather fight poor quality results in the index than fight high quality results who happened to buy links. Maybe Google’s come around to the same philosophy.
- They’re Counting on New Inputs to Help – Part of Google’s initiative in acquiring social gaming companies, building social platforms and making data deals with folks like Twitter could be to help combat spam. They may have hopes that leveraging these new, less polluted (or, at least, more easily trackable) forms of recommendation/citation can be a big win for webspam and search quality.
Why Rant About Spam?
“Blah. Blah Blah. So what if Google’s not doing as much to stop spam as they have in years past?” I hear you ask.
My concern is primarily around the experience of searchers and what it might mean if results become polluted not just by good or relatively good sites that happen to buy or manipulate links, but by really bad crap – the sort that makes searchers want to find a new way of getting information on the web (Facebook Q+A? Twitter? Yelp?). Search today is an amazing marketplace of web builders, marketers, suppliers and customers. If the last of these – the customer – slowly becomes disenchanted with Google, the world of search marketing and the amazing utility of search in general may come to an end.
If you use search engines or work in search marketing, that should be the last thing you want.
That said, if you believe that most of the “spam” will eventually be beaten out by either legitimate results or by better sites that also spam/manipulate links, then there’s much less to worry about (I’m not fully in either camp and can see both sides).
So, What Should Legimitate Marketers Do?
Please DO NOT go out and spam the results, buy links, submit to crap directories and open up link farms. Even with this current trend, I believe that would be terrible advice. Plenty of sites do get caught and filtered, and I’d rather know that my site was safe and every piece of content I added and link I built would help bring more traffic than constantly worry about the small but real risk of being penalized or banned.
One thing Google has done is continue to make the experience of penalization a horrific one. It’s hard to know if you really have a penalty, nearly impossible to figure out what triggered it and onerous, almost Kafka-esque, to attempt to get back into their good graces. If you can live with that risk, as professional black hats do with their churn-and-burn strategies, then it’s less of a concern. But if you’re building a real business, Google is still driving 70%+ of the searches on the web in the US (and 90%+ in many other geographies), and it would be foolish to take such a terrific risk.
As to the question of reporting the spam of your competitors – that’s up to you. However, Google has certainly made it a less likely, less rewarding activity. Nearly every day, we answer PRO Q+A related to the question of link manipulators outranking legitimate marketers and sites, and I can recall only once in the hundreds of questions I’ve answered in the last few years when a spam report actually led to action (to be fair, I don’t follow up consistently on every one, but many of our PRO members will send a regular ping with updates).
What we can do is to re-double our efforts to build great sites with amazing value for people. No matter what the “search” experience of the future is like, those sites and pages that provide a remarkable experience are sure to surface near the top and receive the added benefit of word-of-mouth praise, viral spread and citation in whatever forms it may evolve to, both online and off.
Some Caveats to My Experience
There are millions of queries that are remarkably spam free and Google has done a consistently exception job fighting spam over the years. However, the recent past has me concerned that they are no longer as interested, diligent or capable of combatting even the most basic spam techniques.
It’s also certainly the case that I’m regularly exposed to many queries and topics that SEOs, both black hat and white, focus on, and thus might see more spam than the average searcher (though anecdotally I’d guess they’re seeing more, too).
What Do You Think?
Have you been seeing more results in the rankings that are performing well despite having virtually no “natural” links? Have you seen Google take action on spam reports? Why do you think the recent past has many fewer examples of big spam-cleaning updates?
I’m looking forward to some great discussion – and this week I’ll be at SES San Francisco (on 5 different panels!) – feel free to grab me and chat privately there, too!
p.s. With regards to Bing, the only other major US search engine now that they’re powering Yahoo! (or on the verge), my opinion is that they have been making substantive strides. They’re still behind Google in many areas (and ahead in a few), but at the current rate, we might actually see Bing surpass Google’s spam detection and filtering in the next 18-24 months, though they will probably still be playing catch up in long tail relevancy/quality.
