Can you confirm legitimate Google Bot traffic?
-
We use Cloudflare as a firewall. I noticed a significant number of blocks of bot traffic. One of the things they do is try to block bad bot traffic. But it seems they are mistakenly blocking Google Bot traffic. If you use Cloudflare, you may want to look into this as well. Also, can you confirm if the following IPs are for legitimate Google Bots?
66.249.79.88
66.249.79.65
66.249.79.8066.249.79.76
Thanks,
-
Thank you very much for all of your feedback folks. I really appreciate it. Cloudflare has acknowledged that they are aware of the issue and working to correct it. It also affected Bing Bots. I ended up white listing the IP ranges, and our Google crawl has increased as a result.
-
Dammit, my mistake.
Was proabaly still asleep. I've checked with my log files and updated itThanks Alex!
-
10.0.0.0/8 is a private subnet and therefore unroutable, so I don't think you would have seen traffic from that subnet from Googlebot!
-
What Alex has pointed out is what you're needing.
Also, I'd like to remember you that Googlebot crawls from several IP ranges. Not only the 66.xxx.xxx.xx.
EDIT:
Alex pointed me out that I've made a mistake. So I've checked my data in server logs.
In some of the sites I'm working, most of the traffic comes from the range: [66.249.6x.xxx-66.249.7x.xxx]Always, when in doubt, with a reverse DNSlookup you'll find out whether it's an official GoogleBot or not.
Hope it helps.
Best luck.
Gaston -
Following the advice here: https://support.google.com/webmasters/answer/80553?hl=en I can confirm that they are indeed Googlebot.
If you use Fetch as Google in Search Console, are you seeing errors?
Cloudflare has an article here on the subject:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Https problem on google result.
Hello everyone. My problem is SSL certificate... Send all links to google, after google shows https link no problem. But a few minutes ago my home page link not have an SSL..
Technical SEO | | dalapayal
Please check this page : https://www.bodrumtransfermarket.com Where do I make a mistake? Thanks for all...0 -
Using Google Adwords is good?
I heard about that if you using adwords, google drops your ranking a little bit. Because of you already pay money for results. I think that is reasonable.
Technical SEO | | umutege0 -
Google Trusted Ranking effect?
We are looking into getting the google Trusted badge. We meet the requirements and now just deciding if it is worth the time to do it now or later. Has anyone noticed a difference in your organic rankings or PLAs with it. What about conversion rates? Any input would be great!
Technical SEO | | DoRM0 -
De-indexed from Google
Hi Search Experts! We are just launching a new site for a client with a completely new URL. The client can not provide any access details for their existing site. Any ideas how can we get the existing site de-indexed from Google? Thanks guys!
Technical SEO | | rikmon0 -
Google Indexing
Hi Everybody, I am having kind of an issue when it comes to the results Google is showing on my site. I have a multilingual site, which is main language is Catalan. But of course if I am looking results in Spanish (google.es) or in English (google.com) I want Google to show the results with the proper URL, title and descriptions. My brand is "Vallnord" so if you type this in Google you will be displayed the result in Catalan (Which is not optimized at all yet) but if you search "vallnord.com/es" only then you will be displayed the result in Spanish What do I have to do in order for Google to read this the way I want? Regards, Guido.
Technical SEO | | SilbertAd0 -
Can Google read text in Javascript?
We have just completed the redesign of our product page, which you can see here: http://www.uksoccershop.com/p-19045/2011-12-Chelsea-Adidas-Away-Football-Shirt.html Because we want the select size / add to basket section to appear prominently, you can see we are showing only a snippet of the product description in this section and then user has to click "more" to see it. My question is, can Google read the product description here since it's in Javascript? The code is as follows: 2011-12 Chelsea Adidas Away Football Shirt £44.99 Item Code:379606 Brand new, official Chelsea away shirt for the 2011/12 Premiership season, available to buy in adult sizes S, M, L, XL, XXL, XXXL. This football shirt is manufactured by Adidas and is black in colour.[ More...](javascript:void(0);) Brand new, official Chelsea away shirt for the 2011/12 Premiership season, available to buy in adult sizes S, M, L, XL, XXL, XXXL. This football shirt is manufactured by Adidas and is black in colour. Cheer on the Blues in style in the new adidas Chelsea Away Shirt, featuring a striking blue blocked design on an imposing black background complete with the club crest and adidas logo embroidery across the chest for a great style on or off the pitch. The new Chelsea Away Shirt is designed with adidas' ClimaCool technology to bring moisture away from your skin, keeping you cool, comfortable and performing at your best as you emulate the skills of Frank Lampard, Fernando Torres and John Terry on the pitch. Customise your shirt with Premiership shirt printing for your favourite Chelsea stars or choose your own custom name and number. Adult Football Shirt
Technical SEO | | ukss1984
Short sleeves soccer jersey
Chelsea club crest to left chest
adidas logo and stripes
Print sponsor to centre
ClimaCool technology
Machine washable Product code: 379606 The 2011/12 Chelsea away football kit is released on 7th July 2011. <form name="currenychange" action="http://www.uksoccershop.com/p-19045/2011-12-Chelsea-Adidas-Away-Football-Shirt.html" method="get">
<select class="topselectbox" onchange="this.form.submit();" name="currency" style="float:right;"> <option value="USD">US Dollars</option> <option value="EUR">Euro</option> <option value="GBP" selected="selected">UK Sterling</option> <option value="AUD">Australian Dollars</option> </select>
</form> Available Now [Be the first to ask a question](javascript:void(0); "Ask a Question")
[Be the first to review this product](javascript://) Rating: 5 out of 5 stars <form name="cart_quantity" action="http://www.uksoccershop.com/p-19045/2011-12-Chelsea-Adidas-Away-Football-Shirt.html?number_of_uploads=0&action=add_product" method="post" enctype="multipart/form-data"> Which parts of this is Google going to be able to read? Should we make the product title our H1 header for this page and can it currently read that within the code above? </form>0 -
Our Development team is planning to make our website nearly 100% AJAX and JavaScript. My concern is crawlability or lack thereof. Their contention is that Google can read the pages using the new #! URL string. What do you recommend?
Discussion around AJAX implementations and if anybody has achieved high rankings with a full AJAX website or even a partial AJAX website.
Technical SEO | | DavidChase0 -
Google Penalize?
Hello, I read an statement somewhere which stated: "2 identical URLs linked to 2 different popular key phrases next to each other (on the same website/domain) will lead to a Google penalize. Google knows, that both terms are popular. This means, Google will ignore the links to your site (you'll not have any benefit) and the site you have your links on loses authority." What are your thoughts on this statement? Thank you.
Technical SEO | | micfo0