Can I crawl a password protected domain with SEOmoz?
-
Hi everyone,
Just wondered if anybody has been able to use the SEOmoz site crawler for password protected domains? On Screaming Frog you are prompted for the username and password when you set the crawler running, however SEOmoz doesn't. It seems you can only crawl sites that are live and publicly available - can anyone confirm if this is the case?
Cheers,
M
-
Hi,
I don't think it is possible to crawl any password protectec pages. Password protected pages are usually blocked and bots are unable access those pages.
And yes it is true that only sites that are publicly and live can be crawled.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawl depth and www
I've run a crawl on a popular amphibian based tool, just wanted to confirm... should http://www.homepage be at crawl depth 0 or 1? The audit shows http://homepage at level 0 and http://www.homepage at level 1 through a redirect. Thanks
Technical SEO | | Focus-Online-Management0 -
Cross domain canonical issue
Hello fella SEOs! I have a very intriguing question concerning different TLDs across the same domain. For eg: www.mainwebsite.com, www.mainwebsite.eu, www.mainwebsite.au, www.mainwebsite.co.uk etc... Now, assuming that all these websites are similar in terms of content, will our lovely friend Google consider all these TLDs as only one and unique domain or will this cause a duplicate content problem? If yes, then how should I fix it? Thnx for your precious help guys!
Technical SEO | | SEObandits1 -
How should I set up a domain redirect
A client has 2 domains that he wants to use for the same site. At the moment one domain is just an abbreviation of the main domain (not sure why) as follows: www.mygreatpropertycompany.com
Technical SEO | | davidmaxwell
www.mgpc.com (just redirects to the above) He is complaining that when he searches for 'mpc' there are no results (at all) so I'm wondering what the best approach is.There is currently nothing on the main domain that refers to 'mgpc' in it's abbreviated sense - the only place it's being used is the company's email addresses ([email protected]). The redirect is simply a html file in the root of www.mgpc.com as follows: Is there anything I can do to help him out? (this is one of those 'doing a friend a favour' tasks!) Thanks!0 -
Selling Domain,,,,, OOOOPPPPPPsssss!
Hey Everyone, I think I may have screwed up big time. I have an old 4 letter domain that I've been sitting on for many years. I had no idea what it was worth and ended up putting it up on ebay. I have since been told that it's quite valuable. I just tried to end the auction and ebay won't let me end it. My thinking is that someone here may want to take advantage of my mistake. Even better, let me know if there are any tricks to ending the auction. The domain is http://proa.com If for some reason it doesn't sell, where is a good place to sell it at a fair price? Have a good one. David
Technical SEO | | dmac0 -
Two blogs on the same domain
I have had two blogs on the same domain for a while now, and it just occurred to me that no one else seems to do this and maybe it's even weird. http://www.stadriemblems.com/blog/
Technical SEO | | UnderRugSwept
http://www.stadriemblems.com/scouting/blog/ One is our main blog, and one is for a very concentrated niche of customers. What are your opinions on this? Everything from SEO to best practices, to overall unusual-ness?0 -
Crawling and indexing content
If a page element (div, e.g.) is initially hidden and shown only by a hover descriptor or Javascript call, will Google crawl and index it’s content?
Technical SEO | | Mont0 -
How can i increase my website traffic
Hello, my boss has decide a build website we have more than 12500 products in ourwebsite its mtscellular.com, im new as seo but im confused and need help i want to know how somebody help me to increase my website traffic
Technical SEO | | jimmylora0
