Why does Moz only seem to be crawling a snap shot of the site I am working with?
-
I was wondering if anyone can help? I am working using Moz to help improve the SEO on a website I am working with, the website contains thousands of pages, yet for some reason Moz only seems to be crawling a small snap shot of the website.
I know there are particular pages that I had added a couple of weeks ago - about 300 in total - and none of these were showing on the first crawl, so I did another on-demand crawl and some of these showed up then. Despite this, it says it crawled 700ish pages, but there are getting close to 20-30ish thousand live pages on the site.
Any thoughts and guidance as to why they crawling may be stopping?
-
Hi there,
many thanks for your message. The site is an e-commerce website which currently has just short of 8000 products (each on their own page). These pages are all presented in categories to be able to click on, therefore I would imagine that this is all set up with internal links based on that.
There are numerous products showing, but only 700 pages are coming up in Moz. its weird, because I was expecting some duplication errors on a range that I added, yet these aren't showing in the crawl - but they are definitely there on the site, and under several categories that have links to the categories.
I'm a little stumped at why it doesn't seem to be crawling the full site.
-
Hi there,
Are you internally linking to all these pages that you want Moz to discover? It is possible that those pages are orphaned and they do not have any internal links pointing to them, so Moz cannot crawl them.
Ross
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved What are the top 5 things you recommend you should do when you first subscribe to Moz?
Hi
Getting Started | | djnlowe
Im new to Moz and would like to get the most out of it as quickly as possible. so like the subject say - What are the top 5 things you would recommend I should to get the most out of Moz? #getstarted0 -
Can Moz Monitor a JS Site?
We are building a new site that, on the blog landing page, uses JS to populate the individual blog article links on the page. The links are not viewable in the page source, but do appear once the page fully renders. After running an on-demand crawl of the site (in QA...not indexed yet), it appears that Moz isn't indexing these pages, and it also isn't reading other page elements that load later (like an H1 that is rendered in JS but not in page source). Are we going to be able to use Moz to track this site? Is there some setting to help?
Getting Started | | Rodrigo-DC0 -
Does MOZ pick up every issue in one crawl?
Hi, Does MOZ pick up every error/warning in one crawl? Or does it take numerous crawls? Many thanks Lee
Getting Started | | lbagley0 -
Duplicate Content after Moz Site Audit
Hello folks, So I signed up for the trial version of the Moz tool and ran an initial site audit. One of the site audit results is confusing me.
Getting Started | | jjimen03
It reports that there are two pages with duplicate content ( Each page has a duplicate page with duplicate content in it).
When I take a look at what those pages are, here is what I see: mysite.com/Contact-Us.html
mysite.com/contact-us.html
( The difference in the above is the Contact and Us, the first letters are capitalized on one of the URLS) mysite.com/index.html
mysite.com Now I am confused because for one thing, I don't have 2 Contact Us html files uploaded on my hosting server.
Why is Moz seeing 2 Contact Us pages? How to remove one? Regarding my home page, why is it flagging the same page as two different pages? How to remove of them?0 -
MOZ Monthly Custom Report Question
How do I set the Moz to produce monthly report on 1st of every month. I dont see any option there. we have 50+ Clients in Moz now and we need to generate report on 1st day of all month to send them report. Its 3rd Oct today and I don't see my previous month report being generated. Can you guys help me
Getting Started | | emarketexperts0 -
Moz health check for newbies
Hi I'm on the free trial, I'm finding it really useful. since I was 16 (16years ago) all I've done is cook. After 3 weeks of Moz I feel like I'm or the way to marketing my business instead of having others tell me how to do it. A while back I used to use SEO freelancers to buff my site, but recently I realized that I was not doing enough to promote my business myself, I was trying to pay others to do it for me. I goggled Soe and marketing, of course I found Moz while doing so! The content is exceptional and really goes in depth and we are now coming up with a new content strategy! The problem is that I have this gut feeling that The moz product is worth the money it asks for as I can easily identify the issues or opportunities my site has, I still feel that I lack the competence to use moz to its full potential! Is there an option to have someone skpe me and give me some tips or in sites with in my own campaign? I've spent a lot of time reading the blog content and watching the videos but this all seams to focus on how to get results but not so much on how to use the Moz console/dashboard. I've probably just missed a hole bunch of content but if not do Moz have and option to help set up your Moz analytic and Research so you can get the most relevant info to you? Any help is most welcome Kind regards Ben Horle SussexChef.com
Getting Started | | SussexChef831 -
Crawling Website
How often does Moz crawl the website? I have a lot of high priority crawl issues but my web development team says that it fixed them. However, Moz still shows me that none of the errors are fixed.
Getting Started | | Leoni0 -
Is there a way MOZ can help me get HQ links?
I'm new to MOZ, I'm on the niche sites building. Is there an easy way to find HQ pages to post to with MOZ? Like it's with Market samurai.
Getting Started | | bishop230