Massive increase in Moz crawl.
-
I have a subdomain which has just started to be crawled by Moz, Previously this wasn't the case.
The sub-domain had 16,000+ issues. Why has Moz started to count sub-domains as part of the main domain, has Google started to do this aswell?
-
Hi Dave!
Thanks for reaching out! My name is Anne, and I helping the Mozzer Alliance today! There could be a few different reasons we've just started crawling this subdomain:
1. Did you recently link this subdomain to your site differently? Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. This could explain why we just started crawling the subdomain.
2. Did you change your robots.txt for this subdomain? Our crawler, Rogerbot, follows robots.txt. If you edited this site recently, this could also explain why we just started crawling your subdomain.
I hope this helps! If you have more questions, please reach out to the help team at https://mza.seotoolninja.com/help/contact. From there we can help you troubleshoot what's going on, but keep your campaign details off the interwebz.
Have a great weekend!
Anne
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How get rid of 403 crawl error?
My wordpress website has 162 crawl 403 errors. Based on what I read it means that the server is denying crawlers to access the pages. The pages itself will load so guessing it's just an issue with crawlers only. How do I go about fixing this issue?
On-Page Optimization | | emrekeserr30 -
I am trying to better understand solving the duplicate content issues highlighted in your recent crawl report of our site - www.thehomesites.com.
Below are some of the urls highlighted as having duplicate content -
On-Page Optimization | | urahul
http://www.thehomesites.com/zip_details/76105
http://www.thehomesites.com/zip_details/44135
http://www.thehomesites.com/zip_details/75227
http://www.thehomesites.com/zip_details/94501 These are neighborhood reports generated for 4 different zip codes. We use a standard template to create these reports. What are some of the steps we can take to avoid these pages being categorized as duplicate content?0 -
Crawl errors
Hi I have the following errors on my site and was wondering would it help improve my ranking to fix : Missing Meta Description Tag 137Duplicate Page Title 17Title Element is Too Long 6Temporary Redirect 3
On-Page Optimization | | WallerD0 -
Is the HTML content inside an image slideshow of a website crawled by Google?
I am building a website for a client and i am in a dilemma whether to go for an image slideshow with HTML content on the slides or go for a static full size image on the homepage. My concern is that HTML content on the slideshow may not get crawled by Google and hence may not be SEO friendly.
On-Page Optimization | | aravinn0 -
EMD vs brand? I'm new to moz
Hi to all so this is my first month using moz tools. When I check my competitors and keywords, everyone close to google #1 is a emd site But with google always getting smarter, is it smart in order to compete to get an emd? So say I rank first page #9, or not at all for my brand, if since my site is a construction site, if instead of say abc construction.com I find colorado home builders.com etc would I maybe do better? Or would google give me a bad ranking? I am just so lost? One last question? If I did build a emd site then just pointed my brand to it, would this be good or stupid for seo? thank you for all advice and tips chris
On-Page Optimization | | asbchris0 -
Reducing number crawl-able links?
Hello, I just like to ask for best practice when it comes to reduce number of internal links on a site with a mega menu. Since the mega menu lists all categories and all their subcategories it creates a problem when all categories are linking to all categories directly.. Would the method below reduce the number of links and preventing the link juice flowing directly from category to category? [(link built with JavaScript and the html5 "data-" attribute) Thinking of using these links to categories in the menu not directly below the parent category.](#)
On-Page Optimization | | AJPro0 -
Website pages missing from seomoz crawl
Hi! I just added a website and the crawling result output has only 42 pages but my website has about 75 pages. What am i missing? Thanks!
On-Page Optimization | | ovieira0 -
Why has only one page been crawled?
I went over the list of possible reasons, but I don't think any of them apply to the site in question: sarahhueniken.com. Any other reasons only one page would be crawled?
On-Page Optimization | | scottsemple0