Massive increase in Moz crawl.
-
I have a subdomain which has just started to be crawled by Moz, Previously this wasn't the case.
The sub-domain had 16,000+ issues. Why has Moz started to count sub-domains as part of the main domain, has Google started to do this aswell?
-
Hi Dave!
Thanks for reaching out! My name is Anne, and I helping the Mozzer Alliance today! There could be a few different reasons we've just started crawling this subdomain:
1. Did you recently link this subdomain to your site differently? Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. This could explain why we just started crawling the subdomain.
2. Did you change your robots.txt for this subdomain? Our crawler, Rogerbot, follows robots.txt. If you edited this site recently, this could also explain why we just started crawling your subdomain.
I hope this helps! If you have more questions, please reach out to the help team at https://mza.seotoolninja.com/help/contact. From there we can help you troubleshoot what's going on, but keep your campaign details off the interwebz.
Have a great weekend!
Anne
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How get rid of 403 crawl error?
My wordpress website has 162 crawl 403 errors. Based on what I read it means that the server is denying crawlers to access the pages. The pages itself will load so guessing it's just an issue with crawlers only. How do I go about fixing this issue?
On-Page Optimization | | emrekeserr30 -
MOZ identifies duplicate titles - one has' www' in the title
MOZ has identified duplicate titles - one has' www' in the title. - we have a few pieces of content where the same thing is happening. Not sure how this has happened. Should we do something about this? Will it cause problems for ranking? | KETAMINE GUIDE FOR DRUG WORKERS - free | Harm reduction informationhttp://substance.org.uk/harm-reduction-information/ketamine-guide-for-drug-workers-free | 13 | 2 |
On-Page Optimization | | Substance-create
| KETAMINE GUIDE FOR DRUG WORKERS - free | Harm reduction informationhttp://www.substance.org.uk/harm-reduction-information/ketamine-guide-for-drug-workers-free | 13 | 4 | 1 - 2 of 20 -
Moz Crawl Shows Duplicate Content Which Doesn't Seem To Appear In Google?
Morning All, First post, be gentle! So I had Moz crawl our website with 2500 high priority issues of duplicate content, not good. However if I just do a simple site:www.myurl.com in Google, I cannot see these duplicate pages....very odd. Here is an example....
On-Page Optimization | | scottiedog
http://goo.gl/GXTE0I
http://goo.gl/dcAqdU So the same page has a different URL, Moz brings this up as an issue, I would agree with that. However if I google both URL's in Google, they will both bring up the same page but with the original URL of http://goo.gl/zDzI7j ...in other words, two different URL's bring up the same indexed page in Google....weird I thought about using a wildcard in the robots.txt to disallow these duplicate pages with poor URL's....something like.... Disallow: /*display.php?product_id However, I read various posts that it might not help our issues? Don't want to make things worse. On another note, my colleague paid for a "SEO service" and they just dumped 1000's of back-links to our website, of course that's come back to bite us in the behind. Anyone have any recommendations for a good service to remove these back-links? Thanks in advance!!0 -
Blocking Subdomain from Google Crawl and Index
Hey everybody, how is it going? I have a simple question, that i need answered. I have a main domain, lets call it domain.com. Recently our company will launch a series of promotions for which we will use cname subdomains, i.e try.domain.com, or buy.domain.com. They will serve a commercial objective, nothing more. What is the best way to block such domains from being indexed in Google, also from counting as a subdomain from the domain.com. Robots.txt, No-follow, etc? Hope to hear from you, Best Regards,
On-Page Optimization | | JesusD3 -
Using Subdomains to Increase Keyword Density?
I have a website on which I publish lots of news (sometimes up to 10 posts per day). I have a feeling that somewhat it is diluting the density of some keywords I am targeting. By using subdomains, would I avoid diluting keyword density too much? Of course, I am talking about a reasonable use of subdomains (maybe 1 or 2). What's your opinion?
On-Page Optimization | | sbrault740 -
How much content does Google Crawl on your site?
Hi, We've had a debate around the office where some people believe that Google only crawls the first 150-200 words on a page and some people believe that they priority content that is above the fold and other people believe that all content has the same priority. Can you help us? Thanks,
On-Page Optimization | | mdorville
Matt0 -
I want to check which pages have been crawled
I would like to find out which pages have been crawled by seomoz on my site
On-Page Optimization | | seoworx1230 -
How bad is a negative MOZ rating
I have a wp site that has a negative rank, how bad is this? and is links the only way to convert this to a positive? If its relevant the site is http://rnblog.co.uk Results following websitegrader tool. Website Grade 70 Moz Rank -1 Indexed Pages 255 Traffic Rank 10,932,429 Blog Grade Not Graded Linking Domains 2
On-Page Optimization | | robanewman0