In this week’s Whiteboard Friday, we are going to be going through some different ways you can track down old URLs after a site migration. These tactics can be incredibly useful for new clients that have just performed a redesign with less than ideal preparation.
I’ll be presenting eight ways for you to track down these old URLs, but I would love to see some of your own methods in the comments below. Happy Friday everyone!
Video Transcription
Greetings and salutations SEOmoz fans. My name is Michael King. I’m the Director of Inbound Marketing at iAcquire. I’m also iPullRank on the SEOmoz boards and on Twitter.
So today what we’re going to talk about is eight ways to figure out old URLs after a failed site migration. I know you have this problem. You get a new client, they just redesigned, and you have no idea what the old URLs are. They didn’t do 301 redirects. They have no idea what the social numbers are anymore, and you have no idea where to start. Well, I’m going to show you how.
Now one of the first tactics you want to use is the Wayback Machine. You just put the site in there, the URL, the domain, what have you, and see what it has in that index. Once you get that, you can easily just pull off those URLs on the site through the links using Scraper for Chrome or whatever tool you want to use. You can actually pull down a code and pull them out using Find and Replace, whatever you want to do. That’s just one of the tactics that we’re using.
A lot of times people will also not change or update their XML sitemap. So you can just download that XML sitemap and then open it in Excel, and it puts you in these tables. You can just take that first column and copy and paste it into a text file, open it in Screaming Frog, and then crawl and list mode to see if those URLs still exist. Anything that’s a 404, that’s a URL that you can use, and you can easily map those ultimately to the new URLs on that site.
You also want to use your Backlink profile. When I say that, I don’t want you to essentially use one tool, I want you to use as many tools as possible. So definitely start from Open Site Explorer. Also use Majestic, Ahrefs, whatever you want to use, and collect as much link data as possible. Also Webmaster Tools has your links, so use those as well. Then crawl all those links, all the targets of those links and make sure those pages are still in existence. All the 404s, again, you know these are old URLs that you can then redirect to new pages.
Then you also want to check the 404s from Google Webmaster Tools and map those pages to new pages as well. Then you can also use analytics. So pull your historic analytics from before the site redesign and find all those URLs and see which ones are still in existence. Again, go back to Screaming Frog with list mode and make sure that they’re 404ing or 200ing. The ones that are 200, you don’t have to worry about. The ones that are 404s are the ones that you need to remap.
Then you can also use CMS Change Log. So, for example, when you make a change in WordPress to a URL, there’s a record of that, and you can actually pull those URLs out and use those again for mapping.
Then, for those of you that are a little more adventurous, you can go into your log files and see what URLs were driving traffic before it. Same thing as what you would do with the analytics, but just from a server side standpoint rather than just your click path stuff.
And also social media. So people share these URLs. Any shared URL has equity beyond just link equity. So you definitely want to make sure that you’re pushing those social shared numbers to the right URLs that you’re mapping towards, and I wrote a post on that on Search Engine Watch for how you can do that. But you can use the Facebook recommendations tool. So it’s not really a tool. It’s a demo for widget that goes on your site. But essentially, you can go through this tool and put in the domain name, and it’s going to give you all the shared URLs, all the shared content. The way it comes in the box is it’s 300 pixels tall, but if you expand that to a 1,000 pixels, you’ll see the top 20 pieces of content that were shared. So real easily identify a popular URL that you can then redirect.
Also you can Topsy the same way. If people have tweeted these URLs, you can just put that domain name in there. It’s going to search for them. It’s going to give you all the URLs that Topsy has indexed. You can also use Social Mention, any social listening tool you can use the same way. And then also social bookmarks, so things like Digg, Delicious, and such, look and see what people have actually shared and bookmarked for your site.
So that’s a quick one. Hope you guys found that useful, and I’d love to know how you guys have found this to be worthwhile. So holler at me in the comments down there, and thanks very much. Peace.
