Why does Moz only seem to be crawling a snap shot of the site I am working with?
-
I was wondering if anyone can help? I am working using Moz to help improve the SEO on a website I am working with, the website contains thousands of pages, yet for some reason Moz only seems to be crawling a small snap shot of the website.
I know there are particular pages that I had added a couple of weeks ago - about 300 in total - and none of these were showing on the first crawl, so I did another on-demand crawl and some of these showed up then. Despite this, it says it crawled 700ish pages, but there are getting close to 20-30ish thousand live pages on the site.
Any thoughts and guidance as to why they crawling may be stopping?
-
Hi there,
many thanks for your message. The site is an e-commerce website which currently has just short of 8000 products (each on their own page). These pages are all presented in categories to be able to click on, therefore I would imagine that this is all set up with internal links based on that.
There are numerous products showing, but only 700 pages are coming up in Moz. its weird, because I was expecting some duplication errors on a range that I added, yet these aren't showing in the crawl - but they are definitely there on the site, and under several categories that have links to the categories.
I'm a little stumped at why it doesn't seem to be crawling the full site.
-
Hi there,
Are you internally linking to all these pages that you want Moz to discover? It is possible that those pages are orphaned and they do not have any internal links pointing to them, so Moz cannot crawl them.
Ross
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz campaign for a subdomain with no home page
Hello all, I'm trying to setup a campaign in Moz for one of our subdomains that we use for marketing landing pages. Unfortunately there really is no home/landing page for the subdomain so when I enter it into the campaign settings it says the site isn't accessible. Is there anyway for me to add this? I tried entering the subdomain with a star, e.g. pages.mydomain.com but it would not accept it. Any ideas? Thanks!
Getting Started | | Brando160 -
Open site explorer
Excuse my ignorance but I am very new to all this. I have a new site www.sassandgrace.co.uk and am using MOZ to try and get it right first time. On OSE I am seeing different results for http://www.sassandgrace.co.uk/ and http://www.sassandgrace.co.uk i.e with an without / ) and different results again for http://sassandgrace.co.uk These differences relate to the social page metrics but I would be keen to get one consistent reading. What do I need to do? Also, I know that I have links to the site but none are showing at all. On google console there are a couple showing but I know that there are many more. Is it just a matter of time before they are 'found' or is there something that I should be doing? Sorry for what are probably very basic questions but any help is appreciated.
Getting Started | | Sassandgrace0 -
How Do I Scan My New Site & Grade My Work With The Robots Turned Off? For Pre-Inspection before I launch my Site?
I have a new site that has all the bots turned off so google can't index my site until I'm finished it. I've been working on this site for a couple months now optimizing and I was wondering if there was anyway I can run a preliminary scan on the site for my titles, URLs, Headers, Alt Tags and pretty much anything else that will grade my work and tell me if i did anything wrong? Can MOZ do this with the Bots turned off? Thanks
Getting Started | | Inframan0 -
How to remove campaign from Moz.com
I've accidentally created a duplicate campaign, which I am unable to delete, please help
Getting Started | | shopperlocal_DM0 -
MOZ Starter Crawl not happeneing
Hi I added a new site 48hours + ago and the starter crawler has not even begun collecting data. Any help would be appreciated. cheers Isaac
Getting Started | | sodafizz0 -
Back on Moz
I quick my seomoz subscription a long time ago. I got an invite to the new moz analytics. I don't see anything new at first glance. What's changed?
Getting Started | | joseph11790 -
Whenever I try to access campaigns in moz pro I get an error page
I recently signed-up for a new pro account. As I was adding my first subdomain everything was fine until I was asked to link to GA, when I clicked yes I got this error message: 403 Forbidden Now every time I click on set-up campaign I get taken to a page with nothing but the 403 Forbidden text.
Getting Started | | Toptal0