SEOmoz suddenly reporting duplicate content with no changes???
-
I am told the crawler has been updated and wanted to know if anyone else is seeing the same thing I am.
SEOmoz reports show many months of no duplicate content problems. As of last week though, I get a little over a thousand pages reported as dupe content errors.
Checking these pages I find there is similar content (hasn't changed) with keywords that are definitely different. Many of these pages rank well in Google, but SEOmoz is calling them out as duplicate content. Is SEOmoz attempting to closely imitate Google's perspective in this matter and therefore telling me that I need to seriously change the similar content?
Anyone else seeing something like this?
-
Hi
We see the extreme raise in the duplicate content of our site too. If the sensitivity is adapted, will these graphs come down again?
What is your opinion on how Google sees a webshop with lot's of products and filter options? Our site www.dmlights.com/massive for example can have a lot of filtering but we try to counter this in Webmaster Tools with the URL parameters.
Do you suggest to adapt this for good seo?
Wondering about your opinions. Thanks.
-
Hey Scott,
Again, we're sorry about the odd jump in duplicate content errors!
We just launched a new crawler and it is being extremely sensitive to duplicate content. As of now we are picking up duplicate pages on your domain via:
https clones of URLs
Some pages have a “/” trailing after the URL and some don’t
We are also ignoring some rel=canonical directives
This is an issue that other users are seeing with their crawls. Our engineers have made some changes to the crawler to scale back the sensitivity to these issues on the crawler and you should be seeing the changes within a week or two.
We're really sorry for the confusion.
Best of Luck,
Chiaryn
-
Two good suggestions so far, and both I had checked. Thanks KJ Rogers and Ryan Kent.
This is starting to look like it boils down to how much the new SEOmoz crawler sees content in the same way that Google does.
We did not make any site-wide changes and the URLs identified as duplicate in the report are valid URLs that actually hold similar content (keywords and so forth were changed for each version of a slightly different product through an Excel Concatenate construct to build the content). We have actually seen these pages climb in rank over the months since the content was added.
So, like I said, the sudden identification of these as duplicate by the moz crawler is suspicious to me. Not sure it sees things the way Google does.
-
Without examining your site and the pages involved it is not possible for me to share feedback.
Is it possible you made any recent site wide changes? Changes to your header, navigation, footer or sidebar could have pushed you passed a certain threshhold of duplicate content which triggered a flag.
-
I got the same thing last week. I later found out that mine, using dynamic content on the same page, had speical characters in the url which was taking crawlers to an error page. The error page was showing a list of pages with the url's capitalized. I was able to fix some of them, but it scared the heck out of me.
I had to run a crawl test from SEOMoz to filter out what was going on. Perhaps you have something similar?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Googlebot soon to be executing javascript - Should I change my robots.txt?
This question came to mind as I was pursuing an unrelated issue and reviewing a site's robots/txt file. Currently this is a line item in the file: Disallow: https://* According to a recent post in the Google Webmasters Central Blog: [http://googlewebmastercentral.blogspot.com/2014/05/understanding-web-pages-better.html](http://googlewebmastercentral.blogspot.com/2014/05/understanding-web-pages-better.html "Understanding Web Pages Better") Googlebot is getting much closer to being able to properly render javascript. Pardon some ignorance on my part because I am not a developer, but wouldn't this require Googlebot be able to execute javascript? If so, I am concerned that disallowing Googlebot from the https:// versions of our pages could interfere with crawling and indexation because as soon as an end-user clicks the "checkout" button on our view cart page, everything on the site flips to https:// - If this were disallowed then would Googlebot stop crawling at that point and simply leave because all pages were now https:// ??? Or am I just waaayyyy over thinking it?...wouldn't be the first time! Thanks all! [](http://googlewebmastercentral.blogspot.com/2014/05/understanding-web-pages-better.html "Understanding Web Pages Better")
Algorithm Updates | | danatanseo0 -
Changes in Google "Site:" Search Algorithm Over Time?
I was wondering if anyone has noticed changes in how Google returns 'site:' searches over the past few years or months. I remember being able to do a search such as "site:example.com" and Google would return a list of webpages where the order may have shown the higher page rank pages (due to link building, etc) first and/or parent category pages higher up in the list of the first page (if relevant) first (as they could have higher PR naturally, anyways). It seems that these days I can hardly find quality / target pages that have higher page rank on the first page of Google's site: search results. Is this just me... or has Google perhaps purposely scrambled the SERPS somewhat for site: searches to not give away their page ranking secrets?
Algorithm Updates | | OrionGroup1 -
Domain Change: Leave The Old Domain Homepage Up
We are going to be redesigning our website and switching to a new domain. I think we will set up a permanent 301 redirect from each page of the old domain to a page on the new domain. We would like to leave the old domain homepage up with all content removed and have a link pointing to the new domain. Is there any SEO harm to leaving the old domain homepage up? Thank you! Jessie
Algorithm Updates | | JessieT0 -
How can I use Intuit without getting duplicate content issues
All of my Intuit site show duplicate content on the index pages. How can I avoid this
Algorithm Updates | | onestrohm0 -
Redesign, new content, new domain and 301 redirects (penalty?)
We merged our old webshops into one big project. After a few days we received our rankings back and traffic was coming in. Then suddenly we lost almost all rankings overnight. We did not use any wrong seo techniques and have unique content, written by our own writers. Is this a penalty or do we have to wait longer?
Algorithm Updates | | snorkel0 -
How to build good content and choose right keywords.?
I have started building content for our website using the Wordpress tool. Now I wanted to know that I use GA and the Adwords keyword tool. I go in for exact matching keywords and have selected a few of them. How do I know if these keywords are actually the ones going to give me good traffic? How can I select good keywords and write content along them. I don't wish to over stuff articles with the keywords. How can I refrain from doing so. Any optimum limit through which I know how much of the keyword needs to occur how many times within an article? Please give some good insights as to how this is accomplished? Thanks
Algorithm Updates | | shanky11 -
Google changing case of URLs in SERPs?
Noticed some strange behavior over the last week or so regarding our SERPs and I haven't been able to find anything on the web about what might be happening. Over the past two weeks, I've been seeing our URLs slowly change from upper case to lower case in the SERPs. Our URLs are usually /Blue-Fuzzy-Widgets.htm but Google has slowly been switching them to /blue-fuzzy-widgets.htm. There has been no change in our actual rankings nor has it happened to anyone else in the space. We're quite dumbfounded as to why Google would choose to serve the lower case URL. To be clear, we do not build links to these lower case URLs, only the upper. Any ideas what might be happening here?
Algorithm Updates | | Natitude0