Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why No Goal is recorded in Google Analytics
Hello, I am not sure if i made an error. Can someone please point out. On our sales form, when a user submits the form, the URL displayed is - https://x-y.com/thank-you-2/ I created a Goal like this - 1 ) Under Goal set up, i choose Template option 2 ) Under Goal description, i choose Type > Destination 3 ) and finally, in Goal details field, destination equals to /thank-you-2/ But, no Goal is being tracked. In the first step, should i have selected 'Custom' instead of 'Template' Thanks
Reporting & Analytics | | Johnroger0 -
Google analytics suddenly stopped tracking all my landing pages
Hey guys. I love the new update of GA. Looks so clean. So, of course, I was excited to see how my landing pages were doing. I went to behavior, all content, all pages. And I noticed it's only showing me 19 pages out of the 93 I have indexed. And none of the top ones at all! Can't find them anywhere in GA! Anyone seen this before? Thank you so much
Reporting & Analytics | | Meier0 -
In google analytic for google /cpc it is showing url with 404 which even not exists in my database
Hello All, In google analytic for google /cpc it is showing url with 404 which even not exists in my database that also more than 300 per day. How can it is possible? it is showing /black-friday-offers but I don't have such page. Thanks!
Reporting & Analytics | | pragnesh96390 -
How does Google Maps/G+ traffic show up in Analytics?
Hi Moz Community, I've been trying to figure out how traffic from Google Maps (and G+) shows up in Google Analytics and am struggling to find a good answer online. If someone finds a business through Google Maps and then clicks on the website in the Maps listing, does that show up as a referral from Google Maps? Our site shows virtually zero traffic from Google Maps even though we have a number of listing. Two related questions: if someone clicks through to a G+ page from a Maps result and then visits our website from the G+ page, does that show up in Analytics as a referral from G+? Is traffic from Google Maps or G+ ALSO counted as organic traffic? (Would it be possible to accidentally double-count a visit as both organic and a referral from Maps/G+? Thanks everybody!
Reporting & Analytics | | JohnGroves0 -
Whats 'Other' in Google Analytics (in Acquisition)
When i look in GA under Channels (under Acquisition) 'other' is listed What is 'other' ? I have been told its other unidentified channels as they did not allow 3rd party cookies or surfers were in anonymous/private mode. Other is usually organic traffic that couldn't be identified for the aformentioned reasons. This data is encrypted and available but it violates Google guidelines as they are not allowed to pass personal info//data to third parties so it is automatically filtered. But they are not 'Not Provided' (since that still shows under organic) but is usually/mainly some form of organic visits. Hence Seo can take credit for much of that traffic, is this correct ? Many Thanks
Reporting & Analytics | | Dan-Lawrence0 -
Goal Totals in Google Analytics
How do I get a cumulative tally of all the goals reached in google analytics? I've been using the percentages currently, but I was wondering if there was an easier way. Thanks
Reporting & Analytics | | OOMDODigital0 -
Google Analytics Organic Keyword (Not Provided) Alternative Hacks?
I'm sure everyone is sick of the (Not Provided) keywords because I know I am, and Google reckons that you should only see 10% or less (I'm currently around 13% and growing). I am seeking alternatives or hacks that will help display the keywords? If you have a hack to actually display the keywords, please share it your ideas and thoughts? I recently implemented a hack that will show the landing page of the keywords which is better than nothing. But as an eCommerce site, I have about 10% of my transactions which not provided, knowing the keywords would really help. Would love to hear your thoughts?
Reporting & Analytics | | upick-1623911 -
Expert Google Analytics: store with multiple languages and multiple sub-domians
I hope there's some hardcore Analytics shark out there, with a quick reply 🙂 I am setting up a store with multiple languages for a client and need to be able to track multiple sub-domains as a multiple languages in analytics. Example: dk.somedomain.com -> Danish language
Reporting & Analytics | | ReneReinholdt
www.somedomain.com -> English language
no.somedomain.com -> Norweigen language
.. and so on Now what the client would like is to have one single entity with multiple profiles in analytics, like this: somedomain.com
-> www.somedomain.com
-> dk.somedomain.com
-> no.somedomain.com
ex.. So if the client want's to see stats for the English language then he just vlivks the www profile and in case of danish he clicks the dk profile and so on.
problem here is I can't find an analytics help that addresses this specific issue. I have found this but that doesn't seem to cover it:. Any pointers would be greatly appreciated. 🙂0