Do search engines still index/crawl private content?
-
If you have a membership site, which requires a payment to access specific content/images/videos, do search engines still use that content as a ranking/domain authority factor? Is it worth optimizing these "private" pages for SEO?
-
Technically no, Google will only crawl pages that are publically available.
However, you could use their first click free feature - http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=74536 - depends how much you think you would gain from doing that or what sort of non-payment abuse it opens you up to.
-
If you are using a good pluging or add on to your private area, then No your private content will not be crawled by the Search engines, thus there is no SEO value to Optimize these page.
you can check your pages that is crawled by Google using the "site:"
for ex: Go to google.com and type is the search: site:www.website.com and check if your private area is indexed.
-
Every content that is behind a login is not crawlable, therefore it won't be indexed by search engines. So, no, it is not necessary to SEO those content, even though a basic SEO should have to be done just because it is a way to have a clean site.
For instance, this Q&A will be totally crawlable in two weeks, when it will be visible to all the not PRO members, therefore not behind a login access.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Confused about repeated occurences of URL/essayorg/topic/ showing up as 404 errors in our site logs
Working on a Wordpress website, https://thedoctorwithin.comScanning the site’s 404 errors, I’m seeing a lot of searches for URL/essayorg/topic, coming from Bingbot, as well as other spiders (Google, OpensiteExlorer). We get at least 200 of these irrelevant requests per week. Seems like each topic that follows /essayorg/ is unique. Some include typos: /dissitation/Haven't done a verification to make sure the spiders are who they say they are, yet.Almost seems like there are many links ‘in the wild’ intended for Essay.Org that are being directed towards the site I’m working on.I've considered redirecting any requests for URL/essayorg/ to our sitemap… figuring that might encourage further spidering of actual site content. Is redirection to our sitemap xml file a good idea, or might doing so have unintended consequences? Interested in suggestions about why this might be occurring. Thank you.
Technical SEO | | linkjuiced0 -
If content is at the bottom of the page but the code is at the top, does Google know that the content is at the bottom?
I'm working on creating content for top category pages for an ecommerce site. I can put them under the left hand navigation bar, and that content would be near the top in the code. I can also put the content at the bottom center, where it would look nicer but be at the bottom of the code. What's the better approach? Thanks for reading!
Technical SEO | | DA20130 -
Robots.txt to disallow /index.php/ path
Hi SEOmoz, I have a problem with my Joomla site (yeah - me too!). I get a large amount of /index.php/ urls despite using a program to handle these issues. The URLs cause indexation errors with google (404). Now, I fixed this issue once before, but the problem persist. So I thought, instead of wasting more time, couldnt I just disallow all paths containing /index.php/ ?. I don't use that extension, but would it cause me any problems from an SEO perspective? How do I disallow all index.php's? Is it a simple: Disallow: /index.php/
Technical SEO | | Mikkehl0 -
/forum/ or /hookah-forum/
I'm building a new website on Hookah.org. It will have a forum and blog. Should I put them in Hookah.org/hookah-forum/ and Hookah.com/hookah-blog/ or Hookah.org/forum and Hookah.org/blog I think /forum/ and /blog/ are easier for users but am not sure how much adding the word hookah helps with SEO.
Technical SEO | | Heydarian0 -
301 redirect from root to /index.aspx
I have taken over the SEO for www.domain.net. The way i've inherited the setup is that www.domain.net is 301 redirected to www.domain.net/index.aspx Looking at top pages and linking root domains in Opensiteexplorer I can see that www.domain.net/index.aspx has 1,006 linking root domains www.domain.net has 806 linking root domains. I assume that www.domain.net is passing the value of it's 806 domain links to www.domain.net/index.aspx via the 301 redirect and because of this would expect www.domain.net/index.aspx to be the strongest page on the site and be the url that ranks in the listings for many relevant searches. It appears however that www.domain.net is what is shown in listings and not www.domain.net/index.aspx ?? Can anyone explain why this might be?? If I do a site: search in Google then www.domain.net is indexed and not www.domain.net/index.aspx ??
Technical SEO | | QubaSEO0 -
Advice on display this content on my page for search engines
Hi, my website http://www.in2town.co.uk/Holiday-News is about bringing travel and holiday news to our readers of our lifestyle magazine but i am having problems at the moment with the layout. What i mean by this is, i have written content on the page as an introduction so google knows what this section of the site is about but to be honest it looks rubbish with having the introduction there and i would like to know if i am doing the right thing by having the content there for google to know what my site is about. I have tried taking it away and noticed i dropped in the rankings and when i have put it back up i go up in the rankings, can anyone please give me some advice over this issue
Technical SEO | | ClaireH-1848860 -
How do search engines treat urls that end in hashtags?
How do search engines treat urls that end in hashtags? For example, www.domain.com/abc#xyz.
Technical SEO | | nicole.healthline0 -
Google crawl index issue with our website...
Hey there. We've run into a mystifying issue with Google's crawl index of one of our sites. When we do a "site:www.burlingtonmortgage.biz" search in Google, we're seeing lots of 404 Errors on pages that don't exist on our site or seemingly on the remote server. In the search results, Google is showing nonsensical folders off the root domain and then the actual page is within that non-existent folder. An example: Google shows this in its index of the site (as a 404 Error page): www.burlingtonmortgage.biz/MQnjO/idaho-mortgage-rates.asp The actual page on the site is: www.burlingtonmortgage.biz/idaho-mortgage-rates.asp Google is showing the folder MQnjO that doesn't exist anywhere on the remote. Other pages they are showing have different folder names that are just as wacky. We called our hosting company who said the problem isn't coming from them... Has anyone had something like this happen to them? Thanks so much for your insight!
Technical SEO | | ILM_Marketing
Megan0