Many Duplicate Content Flags
-
Not sure about you all, but I’m loving the new Moz Site Crawler. However, I was noticing that it is identifying a huge amount of pages as duplicate content.
There are about 30,000 pages in this website, with that said we’ve had to make many templates to make the site scalable. Additionally a url rule was lost which caused a significant amount of duplicate pages to be created. I am working through using the moz crawl tool to identify duplicate pages but noticing many pages under “Affected Pages,” are actually unique content pages with initial content that is duplicate.
I read that Moz flags any pages with 90% or more content overlapping content or code. My theory for this is that some templates that are too similar, to the point that Moz reads them as duplicative. Has this happened for anyone else?
In addition, if Moz is flagging these similar pages as duplicate content, do we surmise that Google bots are having the same issue? We have seen issues with rankings as it pertains to the actual duplicate pages but hadn't experienced issues across the unique pages, they are hyperlocal pages so we are able to see rankings quite easily.
-
Hey there,
Sam from Moz's Help Team here!
You're correct - our tool has a 90% tolerance for duplicate content, which means it will flag any content that has 90% of the same code between pages. This includes all the source code on the page and not just the viewable text.
You can run your own checks for percentage similarity using this tool: http://smallseotools.com/similar-page-checker/.
If we're identifying two pages as having duplicate content it's likely that Google will be running into the same issues. You can read a little more about duplicate content, and how to resolve it, on our resource page here: https://mza.seotoolninja.com/learn/seo/duplicate-content
Let us know if we can help with anything else!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I have too many tittle tag issue for my site on moz site crawl error
I have too many tittle tag issue in site crawl error but when I checked manually for the error there is no title in source code. Please Help me to understand
Moz Bar | | Nileshaggarwal0 -
Why do my Moz duplicate content results show me pages with no noticeably similar content?
Sometimes the "Pages with Duplicate Content" results under Content Issues show pages that, from what I'm able to see or otherwise test, have no duplicate content, save for the same navigation that exists on all of my pages. For example, a recent issue said that the following pages had duplicate content:
Moz Bar | | rickmic
https://freezerworks.com/index.php/html/slider-overlay
https://freezerworks.com/index.php/ufaqs/what-do-i-get-with-my-purchase-of-freezerworks
https://freezerworks.com/index.php/videos/fda-and-freezerworks-2
https://freezerworks.com/index.php/lims-testing-module Even a side-by-side of the page source in a text comparison tool shows nothing but navigation and scripts used in every page. Am I not seeing something?2 -
301 Redirects detected as duplicates
I have 19 pages that are all 301 redirected to the same page. Moz is detected these 19 pages a duplication's of each other. Does anyone know how to solve this issue?
Moz Bar | | Worship_Digital0 -
Duplicating a Report to monthly from weekly
Is there a way to duplicate a weekly report and also run it monthly without redesigning it from scratch? I have a great report a client loves but they also want to see it monthly. Thanks! Melinda
Moz Bar | | timesharecmo0 -
OnPage Reports - Duplicate titles and meta descriptions
Hi Moz, I know you guys changed your interface awhile back but I have a question about the new reports. On the old interface, I used to use a report that would automatically run when I created a new account letting me know where the dup titles and meta descriptions were on an entire site. Where can I find this report on the new interface? Thanks Carla
Moz Bar | | Carla_Dawson1 -
Many more 404 being reported in GWT than MA
Hi I have been submitting MA crawl reports to clients developers post going live with a new site migration and instructing them to set up 301's for any 404's still reporting which they have now done (despite instructing them not to go live until all old url's mapped & 301'd to new replacement page, or HP if no replacement). 1) When i look in GWT crawl errors there has been a spike since going live with 660 'page not founds' being reported compared to x11 404's in MA. Could there be a 'lag' in GWT reporting and actually they have already been dealt with just not updated by GWT in this report and the MA report is more accurate ? Should i wait and see/or mark as fixed and see if return tomorrow, or tell dev to immediately investigate ? I have checked some samples links and they are going to 404 type pages so presume they are still broken and urgent issue dev must fix immediately ? 2) How long does it take aprox for the page authority to be transferred via a 301 redirect to a new page since i see some category pages that had good PA and have been 301'd to new category urls, are now showing a PA of just 1 !! Cheers Dan
Moz Bar | | Dan-Lawrence0 -
Duplicate Page Title query in the PRO Campaign tool
Can someone help me on this. I am seeing duplicate page titles on the PRO Campaign Crawl tool on an ecommerce site for example MOZ is saying that these two pages have a duplicate page title: http://www.cheapsnapframes.co.uk/colour-25mm-snap-frames/25mm-green-snap-frame/a0-traffic-green-snap-frame-25mm/prod_1730.html http://www.cheapsnapframes.co.uk/snap-picture-poster-frames/colour-25mm-snap-frames/green-25mm-snap-frame/a0-traffic-green-snap-frame-25mm/prod_1730.html They are the the same product in two categories. When I view the source of both pages the this link is the same in the meta: <link rel="<a class="attribute-value">canonical</a>" href="[http://www.cheapsnapframes.co.uk/colour-25mm-snap-frames/25mm-green-snap-frame/a0-traffic-green-snap-frame-25mm/prod_1730.html](view-source:http://www.cheapsnapframes.co.uk/colour-25mm-snap-frames/25mm-green-snap-frame/a0-traffic-green-snap-frame-25mm/prod_1730.html)" /> So is there something else I need to have done to erradicate this or is it not an issue? Thanks in advance Tracy
Moz Bar | | dashesndots0 -
Can Moz use canconical links to prevent notices about duplicate content issues?
if so how do we enable this - we've an average size site with a few hundred products but they appear in multiple categories, canonical url points to it's primary category (but a new page exists for each section... so for /cat-a/abc there will be another page cat-b/abc and again but the canonical points to cat-a always for that product) basically I see this kind of duplication error / notice as a false positive... help me
Moz Bar | | SEOAndy0