How to allow googlebot past paywall
-
Does anyone know of any ways or ideas to allow Google/Bing etc. to index your content, but have it behind a paywall for users?
-
Thanks Mark,
I have been researching this idea from Google, but it is only for Google News and not Google Web Search.
Also, users would be able to jump the paywall by returning to Google News to search fro more links through to the site.
-
Google has a program called first click free - basically, you need to allow google bot, along with users, to view the first full article they land on. So if you have multiple page articles, you need to give them access to the entire article. After that though, the rest of the content can be behind a paywall.
You can read more about it here - http://support.google.com/webmasters/bin/answer.py?hl=en&answer=74536
And here are the technical guidelines for implementation - http://support.google.com/news/publisher/bin/answer.py?hl=en&answer=40543
Hope this helps,
Mark
-
Not possible. Google's not going to index something that is not accessible to everyone.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Mobilegeddon Help - Googlebot Mobile cHTML & Mobile: XHTML/WML
Our website is (www.billboard.com) and we have a mobile website on a sub-domain (www.m.billboard.com). We are currently only redirecting Googlebot Type "Mobile: Smartphone" to our m.billboard.com domain. We are not redirecting Googlebot Mobile: cHTML & Mobile: XHTML/WML Using this URL as an example: http://www.billboard.com/articles/news/1481451/ravi-shankar-to-receive-lifetime-achievement-grammy, I fetched the URL via Google webmaster tools: http://goo.gl/8m4lQD As you can see only the 3rd Googlebot mobile was redirected, while the first 2 Google bot mobile spiders resolved 200 for the desktop page. QUESTION: could this be hurting our domain / any page that is not redirecting properly post mobilegeddon?
Technical SEO | | Jay-T0 -
Googlebot cannot access your site
Hello, I have a website http://www.fivestarstoneinc.com/ and earlier today I got an emil from webmaster tools saying "Googlebot cannot access your site" Wondering what the problem could be and how to fix it.
Technical SEO | | Rank-and-Grow0 -
Google webmaster tool doestn allow me to send 'URL and all linked pages"
Hello! I made a lot of optimization changes in my site ( seo urls, and a lot more ) , I always use Google Webmaster tools, fetch as Google Bot to refresh my site but now it doesnt allow me to 'Send URL and all linked pages' check the attachment Thank you
Technical SEO | | matiw0 -
Googlebot does not obey robots.txt disallow
Hi Mozzers! We are trying to get Googlebot to steer away from our internal search results pages by adding a parameter "nocrawl=1" to facet/filter links and then robots.txt disallow all URLs containing that parameter. We implemented this late august and since that, the GWMT message "Googlebot found an extremely high number of URLs on your site", stopped coming. But today we received yet another. The weird thing is that Google gives many of our nowadays robots.txt disallowed URLs as examples of URLs that may cause us problems. What could be the reason? Best regards, Martin
Technical SEO | | TalkInThePark0 -
WMT - Googlebot can't access your site
Hi On our new website which is just a few weeks old upon logging into Webmaster tools I am getting the following message Googlebot can't access your site - The overall error rate for DNS queries is 50% What do I need to do to resolve this, I have never had this problem before with any of the sites - where the domains are with Fasthosts (UK) and hosting is with Dreamhosts. What is the recommended course of action Google mention contacting your host in my case Dreamhost - but what do you need to ask them in a support ticket. When doing a fetch in WMT the fetch status is a success?
Technical SEO | | ocelot0 -
Googlebot Crawl Rate causing site slowdown
I am hearing from my IT department that Googlebot is causing as massive slowdown/crash our site. We get 3.5 to 4 million pageviews a month and add 70-100 new articles on the website each day. We provide daily stock research and marke analysis, so its all high quality relevant content. Here are the crawl stats from WMT: http://imgur.com/dyIbf I have not worked with a lot of high volume high traffic sites before, but these crawl stats do not seem to be out of line. My team is getting pressure from the sysadmins to slow down the crawl rate, or block some or all of the site from GoogleBot. Do these crawl stats seem in line with sites? Would slowing down crawl rates have a big effect on rankings? Thanks
Technical SEO | | SuperMikeLewis0 -
I had a massive ogranic traffic drop in the past 72 hours (80%). What should I do? I
I am very very worried as my organic traffic suddenly came to pretty much a halt last Friday. We have been working very hard on content in the last 2 years and have 0 duplicated content with many thousand travel guides, tips and other travel related info. We are also publishing a successful travel blog which used to get quite a bit off traffic. Everything stalled last Friday and since we are operating in the travel industry which is very competitive, this will be very hard on us. In fact, I fear that we will not survive this drop for more than 2-3 months. How do we find out about what happened and what can we do to get our SERPs back? Thanks!
Technical SEO | | madsuh0 -
How much javascript does Googlebot read
We have a site where we have certain navigational links solely for the human user. These links help the user experience and lead to pages that we don't need crawled by googlebot. We have these links in javascript so if you disable javascript these links are invisible. Will these links be considered cloaking even though our intention is not to cloak but save our Google crawl for pages we do want indexed?
Technical SEO | | CruiseControl0