Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Pandora in Google Analytics
Hi Mozers, in Google Analytics our ads are registering a 10 second page duration and a 90% bounce rate, which is pretty atrocious. My questions are: Organic page duration is 3:15 minutes, and Social + Referral are similar. Yet Pandora is bringing the average time on site way down, to less than a minute. Will those paid ad metrics be factored to the User Behavior and therefore negative affect our overall rankings? In other words does Google now see our page duration as less than a minute? The Pandora rep says that Google Analytics and Pandora have "systemic issues" such that GA is very inaccurate with Pandora metrics. I find that difficult to believe. Any insights here? Thanks all! Jane
Reporting & Analytics | | CalamityJane770 -
Is there a way to find out who the admin of a Google Analytics account is?
We have a client who has been searching high and low to gain access to the GA account that is tied to their website, but previous agencies keep stating they don't have access. Is there a way to find out who the admins are on a GA account with the tracking number so we can reach out to that person and ask them to admin the client?
Reporting & Analytics | | marketingmediamanagement0 -
Google Analytics Code
We have a quick question about our Google Analytics code: We recently updated to a re-marketing Analytics code, and some of our traffic numbers seem to be off by a bit. I used the Google Tag Assistant Chrome Extension today, and noticed that it's finding our old Analytics code on our pages, but it's coming up as an error due to "no HTTP Response". I am attempting to remove this code from the website, but it is nowhere to be found in the HTML coding. Only the current one is there. So I'm wondering if this second Analytics code is effecting our traffic and reporting, even if the code is currently non-functional? and if it is, how could I go about removing it if it's not currently in our HTML? Thanks!
Reporting & Analytics | | PlanetDISH0 -
Re-classifying a Traffic Source in Google Analytics
Hey All, I think it's inside of the Admin section of Google Analytics, right now I have a traffic source from the domain of indeed.ca that is being classified as Organic traffic when it should be classified as referring traffic, how can I tell Google Analytics that all traffic from this source should be classified as referral traffic and not organic traffic? Furthermore, after I make this change, will Google Analytics re-classify my past data so I can do a proper analysis? I can't remember how to do this and any help would be appreciated! Thanks!
Reporting & Analytics | | EvansHunt1 -
Google Analytics and Webmaster Tools Setup for Agencies
Hi, As agencies, what are people finding to be the best practices for allowing multiple members of the agency's team to access client WMT and GA data? Have a generic "[email protected]" account that's used for the shares, that anyone in the agency can use as needed (limited, of course, not admin). Have the individual person at the company use their company email for the share for each particular client? [email protected]. Yet what happens when we need someone else to check the GA or WMT data? Any advice is much appreciated.
Reporting & Analytics | | Titan552
Thank you!0 -
Why is Google Analytics showing index.php after every page URL?
Hi, My client's site has GA tracking code gathering correct data on the site, but the pages are listed in GA as having /index.php at the end of every URL, although this does not appear when you visit the site pages. Even if there is a redirect happening for site visitors, shouldn't GA be showing the pages as their redirect destination, i.e. the URL that visitors actually see? Could this discrepancy be adversely affecting my search performance? Example page: http://freshstarttax.com/innocent-spouse/ shows up in GA as http://freshstarttax.com/innocent-spouse/index.php thanks
Reporting & Analytics | | JMagary0 -
Google Analytics set up for non-canonicalized domains
Our client's website is non-canonicalized (www.example.com & example.com load the same thing). Google seems to have made a preference for the www, but canonicalizing to www breaks their Flash website. All we're really trying to do at this time is install Google Analytics for them. What's the smartest way to make sure that both www.example.com and example.com are treated exactly the same by Google Analytics? Google Developers: Domains & Directories states that by default visit data will be separately collected between the two domains, although I found no references to the common www/naked domain issue. In stackoverflow: Does google analytics combine naked domains with the www subdomain? Török Gábor says, "Yes, users will be tracked, but the same visitor coming from www.datalookups.com and datalookups.com will be counted as two different visitors." On the same page, Open SEO says, "This is completely false: www.domain.tld and domain.tld are treaded just the same, and get the same value for the HASH code (the number at the start of each __utm cookie). This an exception: every other subdomain.domain.tld will be handeld as a distinct web site". Can any Analytics experts help me sort this out? Thanks!
Reporting & Analytics | | GOODSIR0 -
Tracking Referring Sources - Google Analytics - Ecommerce
I need some help from anyone that has deal with this issue before.. We are currently running BigCommerce for the shopping cart on chinamist.com. We have enabled e-commerce tracking through Google Analytics and the revenue tracks perfectly. The only issue is that the referring sources are not tracking directly to the revenue being displayed - for the source, all analytics shows is (none). The assumption is that the tracking issue is a result of having a domain and subdomain.. but 'domain and subdomain' was selected when setting up the sites' advanced analytics script. Does anyone have any tips? We've run into a dead end here.. One final note, we're running an Adwords campaign which is showing conversions through the Adwords side of reporting. After integrating the Adwords and Analytics accounts, the analaytics account is still not showing revenue/conversions from Adwords as a source. Appreciate the feedback.
Reporting & Analytics | | Investis_Digital0