Way has the number of pages crawled plummeted?
-
Why has the number of pages crawled for our campaign plummeted in Moz Analytics – down to 729 from over 10k? Don't see any issues in Google Analytics with crawling our site.
-
Wow, that was like an hour of digging--trying to track down the linkscape crawler. I think I've got it now.
-
Rogerbot is the campaign crawler, and OSE has a different name. I know it can be a tad bit confusing at times.
-
Ah huh, I see. So does the campaign crawler have a different name or does it go by Rogerbot too?
-
Chris, what you're talking about is Open Site Explorer. I believe what Eyeglass Guy is talking about is the campaign crawl, which is an entirely separate crawl system.
-
Eyeglasses,
I wonder if your Mozrank or Moztrust has ratcheted down, as well, or if you've made any architectural/navigational changes to the site. With that many pages, it is possible that its not getting crawled as deeply as it had in the past if those dropped pages no longer met the value criteria for crawling by rogerbot--maybe you lost a good link or two or something like that. Also, if you changed any internal linking, it might be that those pages may have been pushed down in the architecture and not be as accessible as they once were.
-
How many pages does your site have? Did you possibly clear up a bunch of parameters and duplicate content, so the crawl is more accurate?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
German blog post with mutated vowel. Page optimization says keyword is not used.
Hey guys! I'm trying to optimize for a keyword that includes a mutated vowel (ä for example). In the URL I simply put it as ae (which most sites that I checked do). For whatever reason it says the keyword is not used on the site at all - which isn't true. Is this a known problem? Haven't found anything in the forums. Thanks for the help. Florian
Moz Bar | | floriannin0 -
Rogerbot will not crawl my site! Site URL is https but keep getting and error that homepage (http) can not be accessed. I set up a second campaign to alter the target url to the newer https version but still getting the same error! What can I do?
Site URL is https but keep getting and error that homepage (http://www.flogas.co.uk/) can not be accessed. I set up a second campaign to alter the target url to the newer https://www.flogas.co.uk/ version but still getting the same error! What can I do? I want to use Moz for everything rather than continuing to use a separate auditing tool!
Moz Bar | | digitalascend0 -
804 : HTTPS (SSL) Error in Crawl Test
So I am getting this 804 Error but I have checked our Security Certificate and it looks to be just fine. In fact we have another 156 days before renewal on it. We did have some issues with this a couple months ago but it has been fixed. Now, there is a 301 from http to https and I did not start the crawl on https so I am curious if that is the issue? Just wanted to know if anybody else has seen this and if you were able to remedy it? Thanks,
Moz Bar | | DRSearchEngOpt
Chris Birkholm0 -
Duplicate page content
The MOZ crawler identifies pages as duplicate content which are not the same.
Moz Bar | | aignerart
The pages http://www.aignerart.com/abstracts-oil-painting/cicli-colora.html and http://www.aignerart.com/abstracts-oil-painting/murs-de-la-ville.html are marked duplicate but they are different paintings. Any ideas?0 -
Domain.com isn't recognized by on-page-grader, but domain.com/index.php is
I am running a website through On-page-grader, as www.domain.com and scores an "F" for a specific keyword. When it's ran as www.domain.com/index.php, it scores an "A" for that same keyword and has everything checked other than "keyword in the domain name". There are no other files such as index.htm, or index.html that would interfere and can't figure out why this page is not being recognized. I checked, the robots and .htaccess file, but do not see anything that would hinder. Could this be a server issue?
Moz Bar | | werkbot0 -
MozBar "Page Load Time" result -140... seconds
Hello all, I have gone through a lot of work to optimize the page size and load time for my site (http://goo.gl/0SYrns) Here is the result: http://www.webpagetest.org/result/140618_6V_PDZ/ And my competitor (http://goo.gl/j7YBcn😞 http://www.webpagetest.org/result/140618_TN_PGP/ According to that tool (and many others) my site is considerably faster than his. However, when I check in the MozBar, the result for mu site is less than 0, around 1 second but his is -1403106240.403 seconds. So how do I get to such a result? I did everything, optimization, cds, etc...
Moz Bar | | branimir.morfov0 -
Duplicate content - Which is the other duplicate page?
Hi I just ran a campaign, and I got a duplicate content warning for some of my pages. When I go into the diagnostic report, I am unable to find the page detected by google as 'duplicate' to the main page. Unless I know which 2 pages are being detected as duplicate, it'll be really difficult to actually solve the problem. Would be great to have any kind of help here. Thanks in advance!
Moz Bar | | rjchugh0 -
Is it no longer possible to export the On-Page grader reports?
Hi Everyone, I was just trying to export the On Page Grader report for my clients website, and found that there is no longer that option. Is this a permanent change with the new branding? Would appreciate any feedback. Thanks very much.
Moz Bar | | Peter.Huxley590