Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can Segment capture organic traffic? If so is it more reliable than Google Analytics?
Hi mozzers, We just learned that our standard GA hasn't been as reliable as we hoped so and we are trying to find other ways to track organic sessions. Which solution would you consider? Is Segment one of them? If so, is it more reliable than Google Analytics? Thanks
Reporting & Analytics | | Ty19861 -
Google Analytics direct traffic dropped by 50% after switching to https
Hi, everyone, have a little dilemma and was hoping to get some advice here. I recently installed a new SSL (3 months ago) to force our URL to start with https://. The site was earlier accessible through https:// and http:// prefix however now I have made it so that all urls are redirected to the correct https:// version. The problem is that direct traffic has dropped by more than 50% after implementing these changes and I can for the life of me figure out why. Why would the direct traffic drop all of sudden after making these changes? I am starting to suspect that the earlier data was inflated and that Google analytics was counting users coming from a http:// version of the URL to a https:// version of the URL was counted as direct traffic. Could that be it? Any other possible causes? Would really appreciate any guidance on this problem. Thanks
Reporting & Analytics | | nsereke0 -
Does Google need Analytics installed to create metrics?
Hi Mozers, Does Google know time on site, number of page visits, bounce rate, etc. even if Google Analytics is NOT installed? Does it measure all that stuff anyway, and if you install the GA code on your site, that is so that YOU can see those metrics? OR can Google only see those metrics if you install GA on your site? Thanks! Jane
Reporting & Analytics | | CalamityJane771 -
Collecting post codes / zip codes in Google Analytics - Terms of Service
Hi Mozzers, Just reading up on Google Analytic's Terms of Service and wondering if collecting post codes / zip codes (from a website's 'Find my nearest...' tool) adhere's to the following: "You will not (and will not allow any third party to) use the Service to track, collect or upload any data that personally identifies an individual (such as a name, email address or billing information), or other data which can be reasonably linked to such information by Google." What do you think?
Reporting & Analytics | | A_Q0 -
Google Analytics SEO Queries Not Showing
Hi All, This might be a silly question, but for all the properties I monitor in Google Analytics, I'm now showing no data for SEO Queries under Acquisition for the past 6 days. Normally I would expect a few day delay in queries, but nothing for 6 days is somewhat peculiar especially as it was functioning fine prior to November 12th. Does anyone have insight into what might be going on? Thanks! URaNMa3
Reporting & Analytics | | amichaels0 -
Google Analytics - Adding a sub-domain
Hi I have a google analytics query.
Reporting & Analytics | | Niki_1
I have a main site with a google analytics tag and I have 2 forms that sit on a subdomain with a different GA code. As I would like to measure end to end tracking, I would like the same GA code on the subdomain. What is the best way for me to implement this? Would I need to make some changes to the GA code that sits on the main site or can I add the the GA code from the main site onto the subdomain? Thanks0 -
Can i know trough Google analytics the most busy time in the day ?
Hello guys,
Reporting & Analytics | | WayneRooney
I have a website with around 600-650 visitor a day.
Can i know trough Google analytics the most busy time in the day ? Thank you0 -
X2 Google Analytics affect page rank ?
Hi there, If you had 2 Google Analytics Accounts one to the main site and another to the blog, could this affect the page position in Google? We've suddenly noticed a drop in our KWs and it was shortly after we added another Google Analytics Account. The blog has 68% Bounce rate and the main site has always been about 48%. Any help would be much appreciated. Many thanks Paul
Reporting & Analytics | | webdesigncwd0