Given that I am currently using a bot sniffer...How can I identify the MOZ bot in order to whitelist it?
-
MOZ is currently blocked from crawling my sites because I use a bot sniffer.
Does anyone know how I can properly identify the MOZ bot in order to whitelist it?
MOZ is using Amazon web services and thus employs thousands of dynamic IPs to crawl.
-
Hi Ryan,
We only block by specific IPs because anyone can spoof a user agent. While i honestly appreciate your effort, could you please put me in touch with someone on the dev side.
Best,
Matt
-
Hey Matt,
Thanks for the question! As Lynn mentioned above, Rogerbot is our user agent, so that is your best option. We use a wide variety of IPs in our crawling that are constantly changing, so there really isn't any IP range that we can provide that will always work.
Cheers,
Ryan Watson
Business Development Associate | Moz -
Hi Matt,
Sorry maybe I misunderstood your question... I thought that was what you are after. How/What exactly is currently blocking the moz bot then?
-
Hi Lynn,
I was under the impression that it's really insecure to whitelist based on useragent.
Thanks,
Matt
-
Hi Matt,
The user agent is called rogerbot, you can see a reference to it here (towards the bottom of the page).
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Keyword Explorer how can I uncover all phrases relating to a single keyword?
From what I can see in Keyword Explorer you cannot enter a single keyword and uncover all search phrases that include this word. Or at least when I try to "Display keyword suggestions that" and select "only include keywords that contain all the query terms" and then enter a keyword. I cannot find a phrase in the list even when I can find it via a different search when if I type it directly.
Moz Bar | | GrouchyKids0 -
Keyword Difficulty Not Working (using it for AU)
Hello Guys, The Keyword Difficulty is not working for AU. Can you please get AU KWs to work? We've: deleted KWs and started over. logged out & logged back in. tried from multiple computers. waited hours. Thanks,
Moz Bar | | WME0 -
MOZ Page Grader - Sorry, but that URL is inaccessible?
Hi, I am trying to use the MOZ page grader on this page - https://www.respuestasparaelhombre.com/datos-sobre-disfunción-eréctil-preguntas but it is saying Sorry, but that URL is inaccessible - any ideas? Thanks
Moz Bar | | Jason_Marsh1230 -
MOZ crawler has been finding a lot of 803 and 804 errors
During last 3 weeks MOZ crawler has been finding a lot of 803 and 804 errors. Meanwhile all pages seem to be working fine. What could cause it?
Moz Bar | | Paruyr0 -
Keyword Difficulty, can I change local and global search volume results from Bing to Google?
Under the Keyword Difficulty tool among the information provided to you when you complete a search is a local and global search volume. Above those numbers it says "Bing Search Volume (Exact Match)" is there any way to change this from Bing to Google? Or do I not understand what exact information they are giving me here?
Moz Bar | | jimmitoro0 -
Moz keywords tool obsolete?
It looks like Google is going to encrypt all user searches, rendering entire sections of SEO tools useless like portions of Moz. What's Moz's reaction to something like this? http://blog.hubspot.com/google-encrypting-all-searches-nj
Moz Bar | | BlueLinkERP0 -
Is it possible to extend my crawling date in SEO Moz?
My web site was crawled by MOZ before week, next crawling date is tomorrow. Because of some reason I am not able to take any action on last week MOZ report.I want to extend MOZ next crawling date, Can I ?
Moz Bar | | ankit.rahevar0 -
Moz reporting appropriate Canonical tag usage but no canonical tag on page !?
I take it this means that the page in question has been referenced via a different pages canonical tag but that the page in question itself does not have a self referencing canonical tag (and that it should do) cheers dan
Moz Bar | | Dan-Lawrence0