How can I clean up my crawl report from duplicate records?
-
I am viewing my Crawl Diagnostics Report.
My report is filled with data which really shouldn't be there. For example I have a page:
http://www.terapvp.com/forums/Ghost/
This is a main forum page. It contains a list of many threads. The list can be sorted on many values. The page is canonicalized, and has been since it was created.
My crawl report shows this page listed 15 times.
http://www.terapvp.com/forums/Ghost/?direction=asc
http://www.terapvp.com/forums/Ghost/?direction=desc
http://www.terapvp.com/forums/Ghost/?order=post_date
and so forth. Each of those pages uses the same canonicalization reference shared above.
I have three questions:
-
Why is this data appearing in my crawl report? These pages are properly canonicalized.
-
If these pages are supposed to appear in the report for some reason, how can I remove them? My desire is to focus on any pages which may have an issue which needs to be addressed.
This site has about 50 forum pages and when you add an extra 15 pages per forum, it becomes a lot harder to locate actionable data. To make matters worse, these forum indexes often have many pages. So if I have a "Corvette" forum there that is 10 pages long, then there will be 150 extra pages just for that particular forum in my crawl report.
- Is there anything I am missing? To the best of my knowledge everything is set up according to the best SEO practices. If there is any other opinions, I would like to hear them.
-
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site Not Indexing & SEOMoz Reporting ZERO On-Page Report Crawls
Any help on this would be MUCH appreciated. One of my sites, aironeairsolutionsinc.com, has recently been rebuilt and the pages tweaked for some basic optimization. Based on my experience, those tweaks (geared toward keywords with relatively low competition locally) usually bump my local sites up into the top 20 or 30 at worst. 3 weeks later, it seems my site is still not indexing with Google. In addition, I AM NOTICING THAT THE ON PAGE REPORTS IN SEO MOZ ARE NOT REGISTERING THAT ANY PAGES ARE BEING CRAWLED. Again, any help from Moz staff would be awesome! :} Thanks, Ricky
Moz Pro | | RickyShockley0 -
How can i extract back link report in OSE of HTTPS domain?
Hello, Recently i was reviewing backlinks of one of the HTTPS domain and was socked to see the difference of backlinks in Majestic SEO and OSE tool. I tried a lot to find the option to explore the backlink of HTTPS domain but i think OSE is not providing that. Can anyone suggest the tool or method to find link report of HTTPS domains? Thank you, Sophie
Moz Pro | | sophiep0 -
Order of urls in SEOMoz crawl report
Is there any rhyme or reason to the order of urls in the SEOMoz crawl report, or are the urls just listed in random order?
Moz Pro | | LynnMarie0 -
Pages Crawled: 0 ?
I've been with SEO Moz for over a month and a half. Why would this weeks crawl have Pages Crawled: 0? I've made no changes since the crawl last week that had 10k pages crawled...
Moz Pro | | mr_w1 -
How to read Crawler downloaded report
I am trying to seperate the duplicate title and description URLs, by looking at the report i am not getting how to find all urls which contain same title and description. Is there any video link on the site which walk me through each part of the report. Thanks, Punam
Moz Pro | | nonlinearcreations0 -
Crawl reports, date/time error found
Hello! I need to filter out the crawl errors found before a certain date/time. I find the date and time the errors were discovered to be the same. It looks more like the time the report was generated. Fix?
Moz Pro | | AJPro0 -
Can I exclude pages from my Crawl Diagnostics?
Right now my crawl diagnostic information is being skewed because it's including the onsite search from my website. Is there a way to remove certain pages like search from the errors and warnings of the crawl diagnostic? My search pages are coming up as: Long URL Title Element Too Long Missing Meta Description Blocked by meta-robots (Which is how I want it) Rel Canonical Here is what the crawl diagnostic thinks my page URL looks like: website.com/search/gutter%25252525252525252525252525252525252525252525252525252525 252525252525252525252525252525252525252525252525252525252525252 525252525252525252525252525252525252525252525252525252525252525 252525252525252525252525252525252525252525252525252525252525252 52525252525252525252525252525252525252525252525252Bcleaning/ Thank you, Jonathan
Moz Pro | | JonathanGoodman0 -
Reducing duplicate content
Callcatalog.com is a complaint directory for phone numbers. People post information on the phone calls they get. Since there are many many phone numbers, obviously people haven't posted information on ALL of the phone numbers, THUS I have many phone numbers with zero content. SEOMoz is telling me that pages with zero content looks like duplicate content with each other.. The only difference between two pages that have zero coments is the title and phone number embedded in the page. For example, http://www.callcatalog.com/phones/view/413-563-3263 is a page that has zero comments.. I don't want to remove these zero comment phone number pages from the directory since many people find the pages via a phone number search. Here's my question: what can I do to make google / seomoz think that thexe zero comment pages is not dupliicate content?
Moz Pro | | seo_ploom0