How do I exclude my blog subfolder from being crawled with my main domain (www.) folder?
-
I am trying to setup two separate campaigns for my blog and for my main site.
While it is easy enough to do through the wizard the results I am getting for my main site still include pages that are in my blog sub folder.
Please Advise!
-
Hi,
I would like to know if this idea, of excluding a subfolder from a campaign, is available now? If so, how do I set it up?
We have a forum in a subfolder and all the data from it kind of blurs my main interest using Seomoz for this website.
-
Hey,
now with the new Moz Analytics platform, has this been implemented yet or still on the plan? I would like to exclude a forum placed in a subfolder as this forum is not our main focus to work with SEO on. It has so many URLs and "eats" up the moz crawl budget for us.
Thanks!
-
Hey Henry,
Thanks for writing in. Unfortunately, there isn't a way to exclude the blog from the crawl for the main site at this time. If you exclude our crawler from those pages, then we wouldn't be able to crawl for the separate blog campaign either. We do plan to allow you to ignore or remove certain pages from your crawl when we switch to the new Moz Analytics platform, but it may be some time before that is available for all of our users. I'm sorry for the inconvenience that causes in the meantime.
Please let me know if you have any other questions.
Chiaryn
Help Team Ninja -
i was just thinking, you might be able to put a robots file in the subfolder telling seomoz bot not to crawl that folder, not sure if that's possible. But make sure you only target seomozs bot only (don't want to tell google not to crawl it).
but that would stop the subfolder being crawled completely, which I don't think you want
-
Yes I am referring to the seomoz bot and I understand that google sees the two sub folders as one domain.
I would like them to be separate just for my own work flow/ability to mange each individually with no spill over.
But if it can't be done it can't be done.
Thank you!
-
I'm guessing you mean the seomoz, bot crawling your site? If so why do you want to crawl them separately, because google won't, they are on the same domain.
to answer your question, I don't think it can be done
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Pages Crawled: 1 Why?
I have some campaigns which have only 1 page crawled, while some other campaigns, having completely similar URL (subdomain) and number of keywords and pages, have all pages crawled... Why is that so? It has been also a while I waited and so far no change...
Moz Pro | | BritishCouncil0 -
Only few pages (308 pages of 1000 something pages) have been crawled and diagnosed in 4 days, how many days till the entire website is crawled complete?
Setup campaign about 4-5 days ago and yesterday rogerbot said 308 pages were crawled and the diagnostics were provided. This website has over 1000+ pages and would like to know how long it would take for roger to crawl the entire website and provide diagnostics. Thanks!
Moz Pro | | TejaswiNaidu0 -
Domain and Page authority dropped on home page
My two strongest links did not show up in the latest Open Site Explorer run. Business.com and BOTW.com...Linking domains went from 21 to 15. Domain dropped 3 pts and Page authority dropped 3 pts I believe. Is there a function in SEOmoz.org that lets you track your domain scores over time? Thanks, Boo
Moz Pro | | Boodreaux0 -
Slowing down SEOmoz Crawl Rate
Is there a way to slow down SEOmoz crawl rate? My site is pretty huge and I'm getting 10k pages crawled every week, which is great. However I sometimes get multiple page requests in one second which slows down my site a bit. If this feature exists I couldn't find it, if it doesn't, it's a great idea to have, in a similar way to how Googlebot do it. Thanks.
Moz Pro | | corwin0 -
How long is a full crawl?
It's been now over 3 days that the dashboard for one of our campaigns shows "Next Crawl in Progress!". I am not complaining about the length... but I have to agree that SEOMoz is quite addictive, and it's quite frustrating to see that everyday 🙂 Thanks
Moz Pro | | jgenesto0 -
Campaigns - domains / sub domains
Dear SEOmoz folk, At the moment, I have one (rather large) site set up as a campain. I have decided to split off parts of my site into sub domains. My question is, can I set up these sub domains as a completely separate campaign on here? I'd very much like the crawl reports separated out for each one. A very real fear I have is that my site is well over double 10,000 pages, so I never truly see the actual number of errors that I have. Thoughts appreciated. Many thanks, Matt
Moz Pro | | Horizon0 -
How long does the seomoz crawl take?
It's been doing it's thing for over 48 hours now and Ive got less than 350 pages... is this norma? It's NOT the first crawl.
Moz Pro | | borderbound0 -
Crawl Issues
My website - qtmoving.com - has 26 articles and when the SEOmoz did a crawl it only found 13 articles. Can someone please give me some insight as to why not all pages are being crawled.
Moz Pro | | CohesiveMarketing0