Moz Crawler not Identifying all Duplicate Pages
-
On two recent site crawls (9/27/14 and 11/4/14) for duplicate content the Moz tool did not ID the following 2 pages, which are 100% duplicate to each other:
http://www.hooksandlattice.com/planter-hampton-241212.html ; Screenshot: http://screencast.com/t/DdwWroUU
http://www.hooksandlattice.com/planter-hampton-721212.html ; Screenshot: http://screencast.com/t/8Lb1cJZmGrhX
As I'm working feverishly to re-write and update the site (goal is ZERO duplicates) I'm finding it challenging to use the Moz tool to get the project done. Does anyone have any feedback or help they can provide for how I can identify all duplicate pages associated with my domain?
Thank you!
Lindsey Pfeiffer
-
Hi Lindsey
Our engineers have confirmed that rogerbot will flag pages that are 100% identical but can sometimes miss pages that are 99% similar. The crawler is deliberately written to err on the side of not reporting false positives which means it sometimes can report false negatives which has occurred in your case. Using a combination of tools such as Webmaster tools can help isolate any pages we have missed.
Hope this helps!
-
Hey Lindsey!
I am not sure why our crawler did not flag those pages as they are 99% identical and are not sharing the same canonical URL. This is very strange and I'll send this up to our crawler engineer to obtain more insight.
Will let you know what I find out once I hear back!
-
Do you check Google Webmaster Tools? Under Search Appearance > HTML Improvements Google will list duplicate titles and descriptions among other things, which might be a help to you.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On Page Grade & Page Optimization Tools not working
Hello! I have an issue with my on page grader and Page optimization. Over the past two weeks i get the attached errors. I searched the forums but after contacting our developer I am in a bit of a standstill. Could someone help on what triggers this? Would highly appreciate it. Best,
Moz Bar | | pranbg
Pran c5CSsHK c5CSsHK0 -
Increase Moz authority above 26
Hi, I have been using Moz for 12 months and found some very useful features. My site’s score steadily increased from 18 to 26. But I seem to have now ‘hit a wall’ and my Moz ranking is static. My question is: Where on Moz.com can I find what next steps I need to take to improve my Moz score? Thanks in advance. Matt
Moz Bar | | wonderdome0 -
How to find the pages in the site which is ranking for zero keyword
I have around three thousand pages in my website,how to find the list of pages which is ranking for zero keywords
Moz Bar | | srinivasan.n1 -
Are we actually getting accurate data on keyword volumes from Moz (or other sources)?
I have a client who does patio furniture repair and restoration. When performing keyword research in Moz for terms like "patio furniture repair" I see that only 11-50 people in the entire US are searching for this term according to the Moz data. However, running an Adwords campaign currently and our top keyword is the phrase match for "patio furniture repair" which has generated over 100 clicks in just a couple of months in ONE county. Is there a better way to research more accurate results on search volume estimates? This makes organic SEO and keyword targeting hard! Thanks, Ricky
Moz Bar | | RickyShockley1 -
Title tag not registering on page optimization
I added a link to the "page optimization" section at moz. When I submit the url to be reviewed (click "optimize") it doesn't find the title tag, although it is definitely there e.g this page
Moz Bar | | digitalalchemy
http://naamanewman.co.il/photosGalleryID.aspx?id=34 Anybody as any Idea why is that? Maybe the tag placement on the DOM (it's not directly under the opening head tag).0 -
Moz Crawl Test Trying to Crawl Contact Form Submit Button Location?
Moz Crawl Test for some reason is trying to Crawl a contact form Widget Submit Location. My obvious guess is that obviously the crawl cannot submit to the required fields…..I believe this because they're only kicking back these errors on the pages I have a contact form widget on. http://crawfordspest.com/pest-control/[email protected] 1412553693 404 : Received 404 (Not Found) error response for page. Error attempting to request page; see title for details. 404
Moz Bar | | Funk-Creative-Media
http://crawfordspest.com/tree-services/[email protected] 1412553693 404 : Received 404 (Not Found) error response for page. Error attempting to request page; see title for details. 404
http://crawfordspest.com/lawn-care/[email protected] 1412553693 404 : Received 404 (Not Found) error response for page. Error attempting to request page; see title for details. 404
http://crawfordspest.com/specialty-services/[email protected] 1412553693 404 : Received 404 (Not Found) error response for page. Error attempting to request page; see title for details. 404 Can you shed any insight to this? I'm a bit worried that I'll have to complete gut the contact form which was one of the major requests my client requested. Or in a worse scenario make all fields not required. It would let so much spam in. I have never seem anything like this at all. But I've learned a lot from Moz, and with major errors like 404 damage Domain Authority greatly. I've fixed 404 issues with newly acquired clients existing sites and tracked through Moz and the domain authority flies up once these errors are fixed. Along with fixing what Webmaster Tools through Google reports back. ..... Let me know if you have any expertise on this matter.0 -
My product pages have no weight / links from root domains with the Moz tool bar
Hi, When I view my home page (http://www.arkwildlife.co.uk) with the Moz toolbar, I see a good PA and a good amount of links from root domains. As I go down the site, it seems to get worse. The category pages (http://www.arkwildlife.co.uk/Category/0/Straight_Foods.html) have a little but not much and then from this point onwards, it's nothing. The product page (http://www.arkwildlife.co.uk/Item/Straight_Foods~Sunflower_Seeds/SUNH/Premium_Sunflower_Hearts.html) is reporting to have no root domain links but I am not sure why. Interestingly, when I click through to a review page (http://www.arkwildlife.co.uk/StockReview/0/SUNH/0/Premium_Sunflower_Hearts.html) it does have some juice. Would anyone be able help on why this is happening and what I need to be looking at in order to resolve it please? EDIT: I've been looking at the hyperlinks and notice something odd. If I review the score with the first link below, it gives a score of 1, but the second gives a PA of 13 with one root domain linked. 1:http://www.arkwildlife.co.uk/Item/Straight_Foods~Sunflower_Seeds/SUNH/Premium_Sunflower_Hearts.html 2:http://www.arkwildlife.co.uk/Item/Straight_Foods%7ESunflower_Seeds/SUNH/Premium_Sunflower_Hearts.html Please note the "%7E" instead of the "~" in the URL. The browser appears to show the ~ character no matter what but the rank of the page changes. I don't understand what the Moz toolbar is doing with this. Note: This behaviour only happens in Mozilla Firefox, in chrome both the rankings are zero for each URL. Many Thanks.
Moz Bar | | nawgie0 -
SEOMOZ On page ranking reports update
I've recently made a load of changes to improve the on page ranking report grades for my site. I was givena C grade for this page: http://www.katalysticevents.com/luxury-wedding-teepee-hire-about/ based on two key phrases Wedding Tipi and Wedding Teepeee. I've made all the changes as advised (excluding H1 tag) by the report but when I rereun the report it's picking up old page data and showing the same problems. Is there some kind of cache thing I'm missing here? The confusing thing is I made the same series of changes for this page: www.katalysticevents.com/luxury-marquee-hire-services/ for Luxury marquee and luxury marquee hire and the results are fine...I've got an A grade. Examples are things like ALT image attirbutes which don't seem to show up on the first page but are fine on the second page.
Moz Bar | | EdoubleD
The site is a Word press site and I'm using the "All in one SEO pack" plugin. Can anyone advise?0