Recovering Your Organic Search Traffic from a Web Migration Gone Wrong

[Estimated read time: 9 minutes]

I know you would never change a URL without identifying where to 301-redirect it and making sure that the links, XML sitemaps, and/or canonical tags are also updated. But if you’ve been doing SEO for a while, I bet you’ve also had a few clients — even big ones — coming to you after they’ve tried to do structural web changes or migrations of any type without taking SEO best practices into consideration.

Whenever this happens, your new client comes to you for help in an “emergency” type of situation in which there are two characteristics when doing the required SEO analysis:

1. You need to prioritize:
   Your client is likely very nervous about the situation. You don’t have a lot of time to invest at the beginning to do a full audit right away. You’ll need to focus on identifying what hasn’t been done during the migration to make sure that the fundamental causes of the traffic loss are fixed — then you can move on with the rest.
2. You might not have all the data:
   You might have only the basics — like Google Analytics & Google Search Console — and the information that the client shares with you about the steps they took when doing the changes. There are usually no previous rankings, crawls, or access to logs. You’ll need to make the most out of these two fairly easy-to-get data sources, new crawls that you can do yourself, and third-party “historical” ranking data. In this analysis we’ll work from this existing situation as a “worst-case scenario,” so anything extra that you can get will be an added benefit.

How can you make the most out of your time and basic data access to identify what went wrong and fix it — ASAP?

Let’s go through the steps for a “minimum viable” web migration validation to identify the critical issues to fix:

1. Verify that the web migration is the cause of the traffic loss.

To start, it’s key to:

• Obtain from the client the specific changes that were done and actions taken during the migration, so you can identify those that had been likely missed and prioritize their validations when doing the analysis.
• Check that the time of the traffic loss coincides with that of the migration to validate that it was actually the cause, or if there were different or coinciding factors that might have affected at the same time that you can later take into consideration when doing the full audit and analysis.

To identify this, compare the before and after with other traffic sources, per device & the migrated areas of your site (if not all of them changed), etc.

Use the “Why My Web Traffic Dropped” checklist to quickly verify that the loss has nothing to do with, for example, incorrect Google Analytics settings after the migration or a Google update happening at the same time.

I’ve had situations where the organic search traffic loss had coincided not only with the web migration but also with the date of a Phantom update (and they had the type of characteristics that were targeted).

If this is the case, you can’t expect to regain all the traffic after fixing the migration-related issues. There will be further analysis and implementations needed to fix the other causes of traffic loss.

2. Identify the pages that dropped the most in traffic, conversions, & rankings.

Once you verify that the traffic loss is due (completely or partially) to the web migration, then the next step is to focus your efforts on analyzing and identifying the issues in those areas that were hit the most from a traffic, conversions, & rankings perspective. You can do this by comparing organic search traffic per page before and after the migration in Google Analytics:

Select those that previously had the highest levels of traffic & conversions and that lost the highest percentages of traffic.

You can also do something similar with those pages with the highest impressions, clicks, & positions that have also had the greatest negative changes from the Google Search Console “Search Analytics” report:

After gathering this data, consolidate all of these pages (and related metrics) in an Excel spreadsheet. Here you’ll have the most critical pages that have lost the most from the migration.

3. Identify the keywords for which these pages were ranking for and start monitoring them.

In most cases the issues will be technical (though sometimes they may be due to structural content issues). However, it’s important to identify the keywords for which these pages had been ranking in the past that lost visibility post-migration, start tracking them, and be able to verify their improvement after the issues are fixed.

This can be done by gathering data from tools with historical keyword ranking features — like SEMrush, Sistrix, or SearchMetrics — that also show you which pages have lost rankings during a specific period of time.

This can be a bit time-consuming, so you can also use URLProfiler to discover those keywords that were ranking in the past. It easily connects with your Google Search Console “Search Analytics” data via API to obtain their queries from the last 3 months.

As a result, you’ll have your keyword data and selected critical pages to assess in one spreadsheet:

Now you can start tracking these keywords with your favorite keyword monitoring tool. You can even track the entire SERPs for your keywords with a tool like SERPwoo.

4. Crawl both the list of pages with traffic drops & the full website to identify issues and gaps.

Now you can crawl the list of pages you’ve identified using the “list mode” of an SEO crawler like Screaming Frog, then crawl your site with the “crawler mode,” comparing the issues in the pages that lost traffic versus the new, currently linked ones.

You can also integrate your site crawl with Google Analytics to identify gaps (ScreamingFrog and Deepcrawl have this feature) and verify crawling, indexation, and even structural content-related issues that might have been caused by the migration. The following are some of the fundamentals that I recommend you take a look at, answering these questions:

A.) Which pages aren’t in the web crawl (because they’re not linked anymore) but were receiving organic search traffic?

Do these coincide with the pages that have lost traffic, rankings, & conversions? Have these pages been replaced? If so, why they haven’t been 301-redirected towards their new versions? Do it.

B.) Is the protocol inconsistent in the crawls?

Especially if the migration was from one version to the other (like HTTP to HTTPS), verify whether there are pages still being crawled with their HTTP version because links or XML sitemaps were not updated… then make sure to fix it.

C.) Are canonicalized pages pointing towards non-relevant URLs?

Check whether the canonical tags of the migrated pages are still pointing to the old URLs, or if the canonical tags were changed and are suddenly pointing to non-relevant URLs (such as the home page, as in the example below). Make sure to update them to point to their relevant, original URL if this is the case.

D.) Are the pages with traffic loss now blocked via robots.txt or are non-indexable?

If so, why? Unblock all pages that should be crawled, indexed, and ranking as well as they were before.

E.) Verify whether the redirects logic is correct.

Just because the pages were redirected doesn’t mean that those redirects were correct. Identify these type of issues by asking the following questions:

• Are the redirects going to relevant new page-versions of the old ones?
  Verify if the redirects are going to the correct page destination that features similar content and has the same purpose as the one redirected. If they’re not, make sure to update the redirects.
• Are there any 302 redirects that should become 301s (as they are permanent and not temporary)
  Update them.
• Are there any redirect loops that might be interfering with search crawlers reaching the final page destination?
  Update those as well.

  Especially if you have an independent mobile site version (under an “m” subdomain, for example), you’ll want to verify their redirect logic specifically versus the desktop one.

• Why are these redirected pages still being crawled?

  Update the links and XML sitemaps still pointing to the pages that are now redirecting to others, so that they go directly to the final page to crawl, index, and rank.

• Are there duplicate content issues among the lost traffic pages?
  The configuration of redirects, canonicals, noindexation, or pagination might have changed and therefore these pages might now be featuring content that’s identified as duplicated and should be fixed.

5. It’s time to implement fixes for the identified issues.

Once you ask these questions and update the configuration of your lost traffic pages as mentioned above, it’s important to:

1. Update all of your internal links to go to the final URL destinations directly.
2. Update all of your XML sitemaps to eliminate the inclusion of the old URLs, only leaving the new ones and resubmitting them to the Google Search Console
3. Verify whether there are any external links still going to non-existent pages that should now redirect. This way, in the future and with more time, you can perform outreach to the most authoritative sites linking to them so they can be updated.
4. Submit your lost traffic pages to be recrawled with the Google Search Console “Fetch as Google” section.

After resubmitting, start monitoring the search crawlers’ behavior through your web logs (you can use the Screaming Frog Log Analyzer), as well as your pages’ indexation, rankings, & traffic trends. You should start seeing a positive move after a few days:

Remember that if the migration required drastic changes (like if you’ve migrated over another domain, for example), it’s natural to see a short-term rankings and traffic loss. This can be true even if it’s now correctly implemented and the new domain has a higher authority. You should take this into consideration; however, if the change has improved the former optimization status, the mid- to long-term results should be positive.

As you can see above, you can recover from this type of situation if you make sure to prioritize and fix the issues with negative effects before moving on to change anything else that’s not directly related. Once you’ve done this and see a positive trend, you can then begin a full SEO audit and process to improve what you’ve migrated, maximizing the optimization and results of the new web structure.

I hope this helps you have a quicker, easier web migration disaster recovery!