Is real google bot like "fetch"or more like "fetch & render"?
-
In GWT we have two options to mimic googlebot visits, "fetch" and "fetch and render", but when the real googlebot visit a page, is he behaving like the former or the latter?
I can see fetch does fetch only the html, while fetch and render does fetch .js and .css as well.
But what does the real googlebot does?
I have checked the web server logs, and I can see the real googlebot sometimes request the .js files too, but not every time it visit a page, sometimes it does, sometimes it does not.
Has anyone figured out when googlebot actually request javascript files?
-
That was my guess, that google somehow cache the .js files, so I changed one javascript file name to a random generated name. Did it last night, I will recheck logs in a couple of days to see what happen.
-
Googlebot is behaving like the latter, "Googlebot constantly crawls and renders URLs across the entire web..." from https://support.google.com/webmasters/answer/6066467, but has the option of streamlining the process by not rendering every time. For example, if many sites are referencing the same centrally hosted code base of js, googlebot doesn't need to constantly recheck that js every time it comes across the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Whatstuffwherebot user agent messing up Google Analytics
Starting yesterday, Aug 26, 2020, I noticed a new bot crawling our site with user agent whatstuffwherebot. Google Analytics is counting these hits as human traffic, completely throwing off my numbers - yesterday, Analytics reported nearly triple my typical number of visitors. As of now, Search Console only shows data through Aug 25 so I don't know if Search Console is also affected. Is anybody else seeing something similar? Does anybody know what the whatstuffwherebot bot is? I don't get any results when I search on Google or Bing. For what it's worth, the traffic is coming from Columbus, OH, running over Amazon AWS via 278 different IP addresses so far. Also, WordFence (my WordPress security plugin) correctly identifies these hits as bot traffic.
Reporting & Analytics | | ahirai0 -
How to track google auto search suggestion click?
Hello Guys, In google.co.uk when I search SEL and google gives me option of different different sites and when I click on any one site then that click tracking I need. I have attached the screenshot to understand easily. Is it possible to track such things or possible via server logs etc? TV99h
Reporting & Analytics | | micey1231 -
Google Shopping tracking as organic traffic - help!
Hey guys, I have an eCommerce client who sometimes runs Google PLA and Google Shopping campaigns. However, we don't run his AdWords for him, just do his SEO. At the moment, the Shopping campaigns track as organic traffic within Analytics. And I can't see any way to change this! It's so infuriating. How do I change this? Thanks!
Reporting & Analytics | | jasarrow0 -
I have few similar job forms that were created for different positions. SEOMoz says, its "duplicate pages". So how do I resolve it? I want my jobs to be searchable in Search Engines.
Hi There, I have few similar job forms that were created for different positions. SEOMoz says, its "duplicate pages". So how do I resolve it? I want my jobs to be searchable in Search Engines. Thanks !
Reporting & Analytics | | pointstar0 -
How Google measure website bounce rate ?
Bounce rate is a SEO signal, but how Google measures it ? There is any explanation about this ? Does Google uses Analytics ? Maybe time between 2 clics in search results ? Thanks
Reporting & Analytics | | Max840 -
My Google Webmaster Search Queries are 0??!!
For the past 2 days Google Webmaster has recorded 0 search queries to my site, but traffic seems stable. Any ideas about why this is happening and what I should do about it? Or is it a Google glitch I should not be concerned about (hoping for this option)
Reporting & Analytics | | theLotter0 -
How long until changes are reflected in Google
I updated my site to remove duplicate titles and also include the rel=
Reporting & Analytics | | MartinSpence46
"next" etc tags. When can I expect to see these reflected in Google andSEOMoz? I also found out I should update my sitemap every month, when will Google pick up the new sitemap? Sorry if these are basic questions, new to the whole world of SEO. Thanks M0 -
Weird drop in ranks... google.fr
Hello folks, Before our domain: <cite>www.convertisseurvideo.net</cite> Keyword: convertisseur video ( convert your video ) is at the first(page) google.fr for this keyword search. Right now, its at second page, really dropped. And our CTR = 1% . How can we improve the CTR? Any toughts ? Thanks. ss.JPG
Reporting & Analytics | | augustos0