Update in Moz spider/tools?? Flagging duplicate content / ignoring canonical
-
Hi all,
Has there been an update in the SEOmoz crawling software?
We now have thousands of dupe content/page title warnings for paginated product page URLs that have correctly formatted canonicals.
e.g.
http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx
... has following pages with identical content that have been flagged:
http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx?p=true&rspage=4
..plus 4 more URL's.
But they all have canonical set. There's even a notice at the bottom of report that tells us there's a canonical set to http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx
What gives, SEOmoz ??
Thanks
Michael
-
Hey Lawrence,
Campaigns have a 95% tolerance for duplicate content. This includes all the source code on the page and not just the viewable text. So if a URL is at least 95% similar in code and content to another URL, this warning will appear.
You can run your own tests using this tool: http://www.webconfs.com/similar-page-checker.php
We don't know what standard Google uses, but it's safe to say they are a bit more sophisticated than us - so you might be okay in this regard as long as you have a couple hundred words of unique text and some unique coding per page. Google won't say how much duplicate content is too much, so we like to be better safe than sorry.
I hope this help. Let me know if you need further assistance.
-Chiaryn
-
Hi Chiaryn,
Thanks for reply and explanation. The different colour-specific pages e.g. Tweed Green and Olive Green have some different content but it's nothing like enough in cases of two greens, two blues etc. as we simplify colour names for search so when there is an Olive and a Tweed Green they both end up having 'Green' as variable in page title, H1 etc. Will fix this.
Do you think the reviews at the bottom of the pages will also trigger dupe content warning? i.e. even if we make all other on-page elements unique for each colour url? (page title, H1, H2, prod description etc) The reviews are quite extensive and are the same on all the separate colour specific product page versions of each style and was thinking today whether we should remove them from these colour product pages (OR perhaps let the colour product pages have their OWN reviews)
http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx
Thanks again
-
Oh, brilliant (re: "See more" aspect) Thanks for the info. Will let you how we tackle this and the repercussions (!) and look forward to hearing how you get on also!
-
Hi Michael,
Thanks for writing in. I already emailed you in response to the ticket you sent in to the Help Desk, but I will copy my answer here for you review.
--
I looked into your campaign and it seems that this is happening because of where your canonical tags are pointing. These pages are considered duplicates because their canonical tags point to different URLs. For example, http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx is considered a duplicate of http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx?p=true&rspage=4 because the canonical tag for the first page is http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx while the canonical for the second URL ishttp://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx, with one URL showing tweed-green and the other showing olive-green.
Since the canonical tags point to different URLs it is assumed that http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx and http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx are likely to be duplicates themselves.
Here is how our system interprets duplicate content vs. rel canonical:
Assuming A, B, C, and D are all duplicates,
If A references B as the canonical, then they are not considered duplicates
If A and B both reference C as canonical, A and B are not considered duplicates of each other
If A references C as a canonical, A and B are considered duplicated
If A references C as canonical, B references D, then A and B are considered duplicates
The examples you've provided actually fall into the fourth example I've listed above.I hope this clears things up. Please let me know if you have any other questions.
--
-Chiaryn
-
We use the "See more" script on our sites, and from what I understand, at least from other Mozzers, this is an okay practice. http://www.seomoz.org/q/using-more-info-javascript-toggledisplay-tag-for-more-info-text
We also use the rel="prev" and rel="next" to some success, but I can't comment on how that's functioning canonical-wise, because IT WAS DROPPED from our latest redesign and is going to be added to our client's website in the latest release. Oye.
I'd love to hear how this works out for you. There are some really great Mozzers on here with loads of experience about canonical tags and duplicate page issues. Can't wait to see what they have to contribute.
-
Hi there,
Thanks for your response.
It's not product page A being seen as a duplicate of product page B etc, but several versions of product A seen as duplicate due to pagination, stemming from reviews for the products that span several pages, so making the rest of the content, titles etc different other than the (crawlable) reviews isn't really an option.
Will look more into "noindex, follow" tags in pagination.
We could have a View All page for indexing showing all reviews (with lots of scrolling!) , with the paginated versions canonicalized to that version (could still serve the paginated version of product page from site navigation perhaps with "noindex, follow" meta tag) Text doesn’t take long to load and this approach would consolidate the review content.
http://googlewebmastercentral.blogspot.co.uk/2011/09/view-all-in-search-results.html
Other option is to use rel=”prev” and rel=”next” implementation which shows Google the relationship between the pages (not sure if it will still be flagged as dupe content in SEOmoz though! Depends if they follow the tag). This way individual pages might get indexed (not sure if that's a good thing?!) perhaps if there's something in a review from (say) page 5 of the product reviews.
http://googlewebmastercentral.blogspot.co.uk/2011/09/pagination-with-relnext-and-relprev.html
Ideally I'd like to implement all reviews on one page and hide them with a facebook-style 'See more' function. Not sure if that counts as hiding content? Will look into this.
-
Hi Michael,
Not sure if this helps you out at all, but I found this about the canonicals and SEOMoz crawl report in a previous Q http://mz.cm/11erRj6:
As far as the SEOmoz crawl reports go, not that setting a canonical won't stop these pages being reported as duplicate content.
From the help:
"Keep in mind that that canonicals will stop the pages from ranking against each other, but they will still show up as duplicate content from a UI perspective, so we will still count them as duplicate."
I have the same issues on my accounts. I'm focusing on making the pages content as unique as possible, or using the "noindex, follow" meta tags to see if that makes a difference.
I know you may have a lot of pages on your website, but perhaps writing short descriptions on your products would help. It might be worthwhile, but completely understandable that it may be a huge undertaking if you have hundreds or thousands of pages.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz tools are returning "url is inaccessible"
Hello everyone, I have been trying to use the on page grader tool and I have also tried to do a site crawl test, and both tools have come back with a "Sorry, but that URL is inaccessible" error. This has not been a problem before. Any ideas why this is happening eg what is blocking it. The url is www.livinghouse.co.uk any help for a novice would be appreciated. PS. I have had another tool also not giving any results, so I assume its something on the site which is blocking the tools. Could this also block Google? Thanks Giles
Moz Pro | | livinghouse0 -
Wild Fluctuations With Yahoo/Bing Moz Report
Whenever I get my weekly custom report, I usually see steady but moderate gains in Google or slight drops 1 to 3 spots. But, with Yahoo/Bing it's just crazy. +25 here, -47 here, etc, etc. Also, we even have keywords that we rank in the top 3 on with google, but not the top 50 with Bing or Yahoo. Not sure how that's possible. Nor do I understand how if Bing powers Yahoo, how we can have reports that show a dramatic increase (or decline) in Bing, but the opposite in Yahoo. Granted, not all the rankings are like that for Bing and Yahoo, but there are so many discrepancies, I'm starting to wonder if it's a software problem or something. Does this happen to anyone else? Thanks, Ruben
Moz Pro | | KempRugeLawGroup0 -
Website data updates
Hello All, I have downloaded some reports for "following" links from SEO Moz and no doubt the report is very comprehensive. However, I know some links listed in the reports does have rel="nofollow" tag attached to it but it is list still shown that they are following links. Can some please guide be to understand why this has happened. Is it possible that SEO Moz has old data of our site because these changes were recently done (2-3 weeks before)..? OR Have I misunderstood following as dofollow..? Any help would be highly appreciated. Thanks, moosa.
Moz Pro | | moosa0 -
Rel="canonical" tag is implemented in my product pages, but still getting canoncal error for products in Moz. What is the problem? me or MOZ?
I have included the rel="canonical" tag in all my product pages, but still getting canonical error in MOZ reports for more than 6 month ! I would like to know if my code is wrong or the MOZ report system is not working properly. Here is an example of my canonical code in line 84 rel="canonical" href="http://www.doornmore.com/slab-single-door-80-fiberglass-courtlandt-1-panel-arch-lite-glass.html" /> Thanks Shayann
Moz Pro | | Shayann0 -
Duplicate page content and title
hi,
Moz Pro | | solutionforweb
i have a serious issue with my site. my website contains 21 pages. but during my weekly report, moz found 84 errors. i.e 42 errors in duplicate page content and 42 errors in duplicate page title... when i see the error in details.. all my 21 links are displaying twice. for example http://domain.com/
http://domain.com/page1.html
http://domain.com/page2.html
and
http://www.domain.com/
http://www.domain.com/page1.html
http://www.domain.com/page2.html so, the same link is repeating twice with www and without www. how do i resolve this error? so please kindly anyone help me....0 -
I want to create a report of only de duplicate content pages as a csv file so i can create a script to canonicalize them.
I want to create a report of only de duplicate content pages as a csv file so i can create a script to canonicalize them. So i get something like: http://example.com/page1, http://example.com/page2, http://example.com/page3, http://example.com/page4, Because I now have to open each in "Issue: Duplicate Page Content", and this takes a lot of time. The same for duplicate page title.
Moz Pro | | nvs.nim0 -
When will the rankings be updated?
Hi we are still waiting for our new rankings, since Tuesday, I understand there is a problem, but when will it be fixed? Thanks. James.
Moz Pro | | Quime0 -
Member Only Content
I run a wordpress based website that contains a large amount of free content, but also a large amount of content that is only accessed via a paid membership. After running a SEOmoz campaign for the site, it showed 3600 errors for duplicate page titles and 1900 errors for duplicate page content. After looking into the errors it became clear that the majority of them were due to the fact that if you clicked on a link to paid content, it would take you to the paid membership sign in page. So how to I go about fixing these errors? I don't want this to hurt my rankings. Or fix it if it already has.
Moz Pro | | CobraJones950