In a recent Whiteboard Friday (12/20/07), Rand talked with Stephan Spencer on the subject of conditional 301 redirects using user agent detection.
I thought it might be useful to briefly cover how easy it is to set up Firefox to use a non-default user agent (like Yahoo, Google, etc) for testing and development. It’s always really useful to have a browser that does this in your armory of SEO tools, especially for spam detection!
The easiest way to switch user agents in Firefox is just to use a user agent switcher. That said, you might not want to install too many plugins to Firefox (or any at all if you need your browsers to stay vanilla), so here’s how to do it with no extra software:
1) Open up Firefox
2) In the address bar, key in about:config
3) Right mouse button click anywhere in the list and select “New” > “String”
4) In the dialog box, key in general.useragent.override and click ok
5) In the “string value” dialog box, key in: Googlebot/2.1 (+http://www.googlebot.com/bot.html), and click ok
Job done! A really quick and easy way to test if your change worked is to see if this website loads a text only version if viewed through Firefox.
It’s amazing how just clicking around can expose all sorts of junk out there. This is still one of my favourite ways to catch spammers! By the way, if you’re looking for a good long list of known user agents, organized by category (spam, user, search engine), then http://www.user-agents.org/ is a brilliant place to go. Very useful when you need to test for a few different search engines. I really love the fact they have an RSS feed for new entries to the database, so if you keep a live database for user agent detection, you can keep it up to date automatically. Nice.
