Does the Moz Pro site crawl, crawl password protected sites?
-
So i asked Moz Pro site crawl to crawl my page, and a lot of issues came up - but for password protected sites. Does the Moz Pro site crawl do this? A lot of the issues, are not relevant for a site that is password protected.
-
Hi there! Tawny from Moz's Help Team here!
Typically requiring a password to access certain pages on the site will prevent our crawler from reaching them, but if those pages are linked to somewhere else on your site, it's possible for our crawler to still find those pages.
This sounds like it might be pretty specific to your Campaign, so I would encourage you to contact us at [email protected] and we'll do our best to sort through this with you.
Thanks!
-
No, none of them are. Most of them are about metadata or content issues. My impression were, that google or Moz for that sake, couldn't crawl sites you have to use a password to access? Thanks for your answer by the way
-
Hi there,
Were the issues that came up for the password protected pages all '403 Forbidden' errors/warnings? If so, I would not be concerned about this assuming that these pages should be blocked with a login. The Moz Pro site crawl will not be able to access these pages or properly analyze them.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Why am I not seeing search.yahoo.com in my site's backlink profile?
When looking at the backlinks for a client's competitor I found that they had a backlink on search.yahoo.com/reviews and our domain (harmonyhit.com) did not. I was using the Link Explorer tool. However, when I go to yahoo Search I see that the website of harmonyhit.com is indexed in Yahoo search and has links to the site. Also, it's sitemap has been recently crawled in Bing webmaster tools which Yahoo uses. Why does our client have the backlink accounted for and we do not?
Link Explorer | | DigitalThirdCoast0 -
Why I am seeing only 2 internal links in Site explorer?
Their are lot of links on my site but site explorer showing only 2 internal links. I don't know why. Can anyone help me please?
Link Explorer | | HSgr1000 -
Moz was unable to crawl your site on Jun 22, 2020\. We were unable to access your site due to a page timeout on your robots.txt, which prevented us from crawling the rest of your site.
Site: www.kpmg.us Getting robots.txt timeout fail since 02/29/20. We've checked our server logs and see no errors. Went through all the steps of the "Troubleshooter". Updated robots.txt to allow rogerbot full access: User-agent: rogerbot
Link Explorer | | KPMG-Search-Social
Disallow: Any ideas how to get roger to crawl my site????1 -
Need help understanding this Moz Chart comparing link metrics against competitors...
Why is my website at 60% and what does it mean? And more importantly, what needs to be done to fix (or not fix) this situation??? Please help... Image of chart attached kcd5r40
Link Explorer | | SamCitron0 -
Crawling 4XX errors because of URL accented
Hello guys, I am experiencing crawling errors in Moz because the URLs read by spiders contain accents and special characters, which I know isn't best but yet my client needs to keep it. I know that Moz "uses percent encoding to parse the HTML in the source code, so any line breaks and spaces in your HTML links or sitemap links are converted to %0A and %20, causing a 404 error". Is there any way to avoid these errors happening in the dashboard? Or am I supposed to simply ignore it?
Link Explorer | | fran8751 -
My site links to my clients not showing in Moz
I've got a web design business with a site that's been out there for 17 years - www.biztoolsone.com. I've got links to my clients' sites in my portfolio page. The backlinks list for my clients, when I check them in Moz, don't show the links from this site. They always showed in the list until a few months ago. The site has a DA of 59 & PA (for home page) of 50. The spam score is 0/17. I'm having a tough time telling my clients that the Moz backlinks list is accurate when that site doesn't show in the list. The same is true for www.fayettevillenc.biz - this is one I created years ago to generate backlinks. It's got a spam score of 4/17, so I'm a little more understanding of that one. Thanks for any guidance!
Link Explorer | | wcksmith10 -
Moz Spam Score 9/17 when there are no links
Hi I was just looking at a competitors new site in OSE and it had a DA and PA of 1 which I would expect and also no links, which again I would expect, basically the guy used to design stuff for me and thinks he can do a better job so he is giving it ago. But he has a spam score of 9/17 but no links - how is this even possible. Thanks Andy
Link Explorer | | Andy-Halliday0 -
Open Site Explorer Recently Discovered Links Question
Hey Mozzers, I have a question regarding the Open Site Explorer Recently Discovered Links. I know that I have receive follow link from a few sites that have more DA than my site. I am wondering why it is taking so long for OSE to discover these links? I double checked the sites to make sure that they don't have NOFOLLOW (sorry for the double negative), and the sites have linked to my site for over a month now. Any ideas? I'm not concerned about it because Search Engines have discovered those links. I am just curious why this is, or if it is normal. Cheers!
Link Explorer | | evan890