Rogerbot will not crawl my site! Site URL is https but keep getting and error that homepage (http) can not be accessed. I set up a second campaign to alter the target url to the newer https version but still getting the same error! What can I do?
-
Site URL is https but keep getting and error that homepage (http://www.flogas.co.uk/) can not be accessed. I set up a second campaign to alter the target url to the newer https://www.flogas.co.uk/ version but still getting the same error! What can I do? I want to use Moz for everything rather than continuing to use a separate auditing tool!
-
It will probably help if you delete the following line from your robots.txt file:
User-Agent: rogerbot
Disallow: /Dirk
PS You also block /js/ for all bots - Google doesn't really recommend that
-
I just check and see that http is 301 to https. But https works.
Can you mail this issue to [email protected]?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Not sure where this url has come from
can anyone please let me know why this has happened on my site. I have just done a crawl test and it comes back with the following <colgroup><col width="576"></colgroup>
Moz Bar | | in2townpublicrelations
| http://howtodrinkless.com/web/20150709201150/http:/www.howtodrinkless.com/ |0 -
Onpage Grader not Finding Keyword in URL
I've noticed that the Onpage Grader is not including my keyword in the URL when the keyword is in the domain. If I grade an inner page and the keyword is in the sub-directory, it finds it. Is this intentional? If so, why does the grader not include my keyword in the domain as Keyword in the URL?
Moz Bar | | Dino640 -
The Moz Spam Score tells me my site has too few backlinks for such a large site. How many links per page would I need to not trigger this filter and stop appearing spammy?
Hello! One of my sites is triggering the 'too few backlinks for large site' filter. I am wondering how many backlinks I need so as not to trigger this. Many thanks for your help. Toby
Moz Bar | | T0BY0 -
Signed up for moz reports - have received Moz error report - need someone who is capable to take report and perform cleanup edits within Joomla site?
Looking for someone in the US - please contact me at [email protected] If available and interested in task. Thanks Mary
Moz Bar | | PortlandWebDesign0 -
Crwal errors : duplicate content even with canonical links
Hi I am getting some errors for duplicate content errors in my crawl report for some of our products www.....com/brand/productname1.html www.....com/section/productname1.html www.....com/productname1.html we have canonical in the header for all three pages <link rel="canonical" href="www....com productname1.html"=""></link rel="canonical" href="www....com>
Moz Bar | | phes0 -
Weird back link showed in moz crawl
Some time ago somebody from this site: http://dianibeach.com created a weird link to our site which had on the end db. Later we have realized that the link was coming from every footer on each page. I believe that the back links from footer does not have realy value and even the more of them the less value. We have asked the guy to remove that links as I thought it might harm our site more then help. Now I I was very surprised to find this link in moz crawl error as second top page on our site in current index??? Can somebody explain how is this possible?? The most ridiculous thing is that when I click on that link it realy opens our site! How is that possible, what is it? This is the link: http://villasdiani.com/?db Thank you very much for any help with this
Moz Bar | | Rebeca10 -
Is there a problem with New Campaign crawls being slow?
Hi I've created 3 new campaigns but results not showing as usual. I know the full results take up to 7 days but I usually see some basic info in a few minutes of creating the campaign. This problem happened a while ago and the campaign was never quite right in SEOMoz. Thanks for your help Steve
Moz Bar | | stevecounsell0 -
Given that I am currently using a bot sniffer...How can I identify the MOZ bot in order to whitelist it?
MOZ is currently blocked from crawling my sites because I use a bot sniffer. Does anyone know how I can properly identify the MOZ bot in order to whitelist it? MOZ is using Amazon web services and thus employs thousands of dynamic IPs to crawl.
Moz Bar | | Felix_LLC0