Crawl diagnostics incorrectly reporting duplicate page titles
-
Hi guys,
I have a question in regards to the duplicate page titles being reported in my crawl diagnostics. It appears that the URL parameter "?ctm" is causing the crawler to think that duplicate pages exist. In GWT, we've specified to use the representative URL when that parameter is used. It appears to be working, since when I search site:http://www.causes.com/about?ctm=home, I am served a single search result for www.causes.com/about. That begs the question, why is the SEOMoz crawler saying there is duplicate page titles when Google isn't (doesn't appear under the HTML improvements for duplicate page titles)? A canonical URL is not used for this page so I'm assuming that may be one reason why. The only other thing I can think of is that Google's crawler is simply "smarter" than the Moz crawler (no offense, you guys put out an awesome product!).
Any help is greatly appreciated and I'm looking forward to being an active participant in the Q&A community!
Cheers,
Brad
-
Glad I could help, Bradley. Let us know if you need help with anything else.
-
Thanks for the thorough response Chiaryn! I figured as much but wanted to make sure I wasn't overlooking anything.
-
Hey Bradley,
You're right; Google's crawler is way more sophisticated than ours is because they have a lot more resources, be they engineers or finances, to pour into their crawler. We think our crawl provides tremendous value and is an excellent way to discover and understand the architecture of your site at scale, but it's not that strange that it wouldn't line up with exactly what a site: search reveals. We also don't always know how Google (or other search engine bots) is going to consider a set of pages, so we would rather be safe than sorry with the data we provide.
Since the page http://www.causes.com/about?ctm=home is linked to from another page on your site (www.causes.com) and resolves with a 200 status, our crawler sees it as an individual page and won't associate it with the main /about page. Instead, it just compares the code and content with the other pages we've crawled and reports back when we find duplicates.
I hope this helps clear things up. Please let me know if you have any other questions.
Chiaryn
Help Team Ninja
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved How does Moz compile the "Important pages on your site are returning a 4xx error!" report?
I have over 200 links in this report (mostly from a staging site). I have deleted that staging site and I cannot find the reference to the other links. So my question is, where is Moz finding these links?
Moz Pro | | nomad_blogger0 -
How to find missing or incorrect title tags with a site hosting lots of pages.
i have a website that features more than 9,000 pages. i'm trying to figure out which ones have missing or incorrect title tags. Should I start with screaming frog??
Moz Pro | | SapphireCo0 -
Is there a tool to crawl website meta tags to locate any in an incorrect language?
Example: I'm interested in crawling a regional site for Brazil to find any meta tags that are still in English with the goal being to fix any localization issues.
Moz Pro | | mattsolar0 -
Duplicate Page Titles & Content
We have just launched a new version of a website and after running it through SEOMOZ we have over 6000 duplicate title & content errors. (awesome) 😕 We have products that show up multiple times under different URLs however we "thought" we had implemented the rel=canonical correctly. My question is - do these errors still show up in SEOMOZ despite the canonical tags being there OR if they were "correct" would we be getting "zero" errors?
Moz Pro | | ZaddleMarketing0 -
I've got quite a few "Duplicate Page Title" Errors in my Crawl Diagnostics for my Wordpress Blog
Title says it all, is this an issue? The pages seem to be set up properly with Rel=Canonical so should i just ignore the duplicate page title erros in my Crawl Diagnostics dashboard? Thanks
Moz Pro | | SheffieldMarketing0 -
Duplicate page content showing up with proper use of canonical tag
Hi, In the Crawl diagnostics reports, I'm getting lots of duplicate errors warnings e.g. duplicate page title. In most cases these are tracking urls and the page has a canonical tag pointing to the original page. It would be helpful if the crawl analysis reports could separate these out from ones that are of genuine concern. It can also happen when there's a noindex tag on a page. Thanks, Leigh
Moz Pro | | Leighm0 -
Crawl Diagnostics Report Lacks Information
When I look at the crawl diagnostics, SEOMoz tells me there are 404 errors. This is understandable, because some pages were removed. What this report doesn't tell me is how those pages were discovered. This is a very important piece of information, because it would tell me there are links pointing to those pages, either internal or external. I believe the internal links have been removed. If the report told me how if found the link, I would be able to take immediate action. Without that information, I have to go so a lot of investigation. And when you have a million pages, that isn't easy. Some possibilities: The crawler remembered the page from the previous crawl. There was a link from an index page - i.e. it is in the database still There was an individual link from another story - so now there are broken links Ditto, but it in on a static index page The link was from an external source - I need to make a redirect Am I missing something, or is this a feature the SEO Moz crawler doesn't have yet? What can I do (other than check all my pages) to discover this?
Moz Pro | | loopyal0 -
All seo reports
I want to find out what all the report's correct definition and when I see improving or going down what does it mean for eg. Organic search report? Keywords? or non-paid keywords?
Moz Pro | | ITWEBTEAM0