Here are some sections of the site that Shopify will disallow crawling in:
-
Admin area
-
Checkout
-
Orders
-
Shopping cart
-
Internal search
-
Policies page
However, as Shopify stores get bigger and more customized, there’s a greater chance that you might need to adjust the robots.txt file. Fortunately, as of June 2021, Shopify now lets you update the robots.txt file.
In order to edit the Shopify robots.txt file, store owners must create a robots.txt.liquid file and then create custom rules to specify any changes.
In order to create a robots.txt.liquid file, store owners can perform the following steps:
-
Login to your Shopify admin area
-
In the left sidebar, go to Online Store > Themes
-
Choose Actions > Edit code
-
In “Templates”, select the “Add a new template” link
-
Find the left-most dropdown and choose “robots.txt”
-
Choose “Create template”
This should create your Shopify robots.txt.liquid file. You can then add rules to your robots.txt.liquid file by adding liquid code. Fortunately, this code isn’t too difficult to add, and Shopify does a good job of highlighting how to do it in their official documentation. Following these steps should allow you to have much more control over which URLs are crawled in your Shopify site.
Sitemap.xml
By default, Shopify will generate a sitemap.xml index file at the URL path “domain.com/sitemap.xml”. Shopify’s sitemap.xml index file will automatically create links to child sitemaps that contain URLs of the following page types:
-
Product Pages (sitemap_products_1.xml)
-
Collection Pages (sitemap_collections_1.xml)
-
Blog Posts (sitemap_blogs_1.xml)
-
Marketing Pages (sitemap_pages_1.xml)
This sitemap.xml file will dynamically update as new pages are added/removed from to the site. Generally, the Shopify sitemap.xml is good to go out of the box and doesn’t need to be adjusted.
One thing to be aware of is that Shopify will include any published pages in the sitemap.xml file. The most common issue we see is that legacy pages that are published but no longer linked to on the site get included in the sitemap.xml file. It’s worth crawling your sitemap.xml to find any instances of published pages that are included in the sitemap but are not important for search engines to crawl.
Adding the “noindex” tag
While you cannot adjust the robots.txt, Shopify does allow you to add the “noindex” tag. You can exclude a specific page from the index by adding the following code to your theme.liquid file.
{% if template contains ‘search’ %}
{% endif %}
As well, if you want to exclude an entire template, you can use this code:
{% if handle contains ‘page-handle-you-want-to-exclude’ %}
{% endif %}
Redirects
Shopify does allow you to implement redirects out-of-the-box, which is great. You can use this for consolidating old/expired pages or any other content that no longer exists. You can do this by going to:
-
Online Store
-
Navigation
-
URL Redirects
The big thing to keep in mind is that you will need to delete a page before you can implement a redirect on Shopify. This means that you’ll want to be really sure you’re not going to use the page in the future. To make this process a little less stressful, we recommend implementing the “Rewind Backups” app.
Log files
As of now, Shopify does not allow you to access log files directly through the platform. This has been confirmed by Shopify support.
Fast Simon implementation
Fast Simon is an enterprise solution that adds robust personalization features to your Shopify store, and is becoming increasingly popular. If your Shopify site is utilizing the Fast Simon technology, you’ll want to be sure that you’re taking steps to adjust any potential indexing issues from an improper implementation.
Confirm that Fast Simon is pre-rendering your website’s content so that Google doesn’t run into crawling and indexing issues. This will give Googlebot a server-side, rendered version of your site that will make it easier for it to interpret the content. For more details, you can read our case study here.
Structured data
Audit existing structured data implementation
Before doing anything with regards to structured data, you’ll want to first audit how schema is being deployed across your site. By doing this, you’ll gain a much better understanding of what changes you need to make to your schema implementation.
When working with Shopify sites, we’ll often find that — upon first investigation — there are multiple instances of structured data. This is because Shopify sites will many times insert structured data from elements such as the theme and third-party apps. The result is that the pages present multiple structured elements as opposed to a single consolidated one.
In the example below, you can see a site that presents two instances of “Product” structured data.
