Few weeks ago we were discussing 302 redirects in Magento as it’s one of the most common questions we receive. But there is one more issue hunting thousands of Magento store owners. If duplicated content is the first thing that came into your mind, you are right. Here, in MavenEcommerce, we audited hundreds of Magneto websites during past few months and more than 95% of them had issues with duplicate content. While Magento is recognized to be a market leader when it comes to SEO optimization, duplicated content remains it’s weak spot. In most cases duplicates occur because of:
- Product filtering
- Product sorting
- Pagination
- Same product in different categories
- Variation of the same product
1. Product filtering duplicates
By default, Magento creates dynamic pages on category/subcategory pages when you filter the products by attribute. For example you decided to filter products by color. It looks like this www.myawesomewebsite.com/cute-ponnies?color=pink. It’s obvious that we don’t want Google to index this page. By the way, you can check if any of such pages are indexed by typing in Google search “site:yourdomain.com inurl:yourparameter(color in our case)”. I also use Screaming frog to check availability of such pages for crawlers(if Screaming Frog find them, Google might as well).
In order to prevent these URLs from being indexed you can use the following methods:
Google Webmaster Tools
Go to Crawl=>URL parameters in your Google Webmaster Tools account. Click “Configure URL Parameters”. Now you see all(or almost all) parameters used by your website. Find parameter that you don’t want to be indexed and in column “Crawl” change status from “Let Google bot decide” to “No URLs”. You can also add parameters that are not listed by Google and set Crawling options.
After changing Crawl options, have a deep breath and be patient, it may take a while while Google de-index URLs you blocked. You also can delete them manually at Google Webmaster Tools-Google Index-Remove URLs but it’s not a good idea if you have hundreds or even thousands of such pages.
Meta Robots Tag
You can also avoid indexing pages with applied filter by adding Meta Robots Tag(noindex, follow) into the head section of parameter based URL. If you decided to use this method, I also recommend adding rel=canonical tag to the header.
2. Product sorting duplicates
This kind of duplicates is pretty similar to Product Filtering duplicates. When you use sorting on category/sub category pages you’ll get URL like this www.myawesomewebsite.com/cute-ponnies?dir=asc&order=name. To prevent such pages from indexing you can use Google Webmaster Tools, Meta Robots Tag or rel=canonical.
3. Pagination duplicates
No doubt that your Magento website is huge and you have a lot of products. Even if you don’t, your products still placed on the pages with pagination options. Basically these pages look like this:
www.myawesomewebsite.com/cute-ponnies?page=2 www.myawesomewebsite.com/cute-ponnies?page=3Keep in mind that pagination duplicates may occur not only at category/sub category pages. Pagination in blog section also leads to duplicates.
How to find out if these pages are indexed? Just search in Google “site:yoursite.com inurl:page”. Got search results with pagination? Fix it ASAP! Didn’t get any results? No one is safe in terms of pagination so Keep reading!
There are two widespread methods to fix pagination duplicates. a)Canonicalization b)Using rel=”next” and rel=”prev”
Canonicalization
You already now how it works, right? So just place rel=canonical from pages www.myawesomewebsite.com/cute-ponnies?page=2, www.myawesomewebsite.com/cute-ponnies?page=3 and point them to www.myawesomewebsite.com/cute-ponnies. Done! You’re safe now!
Using rel=”next” and rel=”prev”
This method was created by Google(Google loves you). The idea is that all paginated pages are connected with each other like this: You need to put
in the head section of www.myawesomewebsite.com/cute-ponnies
in the head section of
www.myawesomewebsite.com/cute-ponnies?page=2and
in the head section of
www.myawesomewebsite.com/cute-ponnies?page=3and so on. It needs a bit of practice =)
Variation of the same product
Let’s say you sell flowers and you have separate page for each color. For example: www.myawesomewebsite.com/beautiful-rose/red-rose and www.myawesomewebsite.com/beautiful-rose/white-rose But your characteristics, description, layout are the same. The only difference is a picture. In such a case Google will treat these pages as duplicates. Fortunately, there are there are three great ways to avoid this:
- Create one page for all variations of your product. I’ve heard that it’s called configurable product=). In this case you’ll have one unique page.
- Just choose one page and put rel=canonical to it from all variations. Here your can practice with rel=canonical again.
- Write unique content for each variation page! This is a difficult way but it’s the most rewarding in term of SEO optimization.
Same product in different categories
It happens all the time. You can find the same product in several categories.
www.myawesomewebsite.com/new/product www.myawesomewebsite.com/popular/product www.myawesomewebsite.com/jewelry/product
Plus, Magento creates top level product URL by default www.myawesomewebsite.com/product. This causes full duplicate as the product is the same. So how to fix it?
Canonicalization
It is the most common solution. Just choose the main page (usually www.myawesomewebsite.com/product) and point all other pages to it.
Remove category
Go to System => Configuration => Catalog => Search Engine Optimization and select ‘No’ in Use Categories Path for Product URLs. I strongly recommend placing 301 redirect for old URL variations(with categories in it).
Use only one category in URL
If you have the same product in several categories, you can choose which category to use in the URL from now on. It can be done by different extensions. I recommend Unique Product URL from Amasty.
If you have any questions about anything I’ve mentioned in this blog post or anything else related to Magento, please feel free to drop me a message via this form.
useful article and surely to apply on our websites
Really informative, thanks for sharing this!
Very Helpful, I think I have started applying from right now.
Duplicate content can be a problem, but thanks sharing such an article.
You’re welcome!
Thanks for sharing. I really loved reading your blog. It was very well tailored and easy to understand. Unlike other blogs I have read, which are really not that great.
Hi Alida,
We appreciate your feedback. Thank you!