Rogerbot will not crawl my site! Site URL is https but keep getting and error that homepage (http) can not be accessed. I set up a second campaign to alter the target url to the newer https version but still getting the same error! What can I do?
-
Site URL is https but keep getting and error that homepage (http://www.flogas.co.uk/) can not be accessed. I set up a second campaign to alter the target url to the newer https://www.flogas.co.uk/ version but still getting the same error! What can I do? I want to use Moz for everything rather than continuing to use a separate auditing tool!
-
It will probably help if you delete the following line from your robots.txt file:
User-Agent: rogerbot
Disallow: /Dirk
PS You also block /js/ for all bots - Google doesn't really recommend that
-
I just check and see that http is 301 to https. But https works.
Can you mail this issue to [email protected]?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Monthly Keyword Volume Differences Keyword Research V's Campaign Rankings
Why are there differences in the keyword volume data, is this a UK only issue? Am I missing something? Within Keyword Research;
Moz Bar | | GrouchyKids
suspended ceiling systems swindon (0-10) Within Campaign Rankings:
suspended ceiling systems swindon (no data) It's not just one instance either its across multiple keywords.0 -
The factors considered in the new domain authority algorithm? On-site factors we can use to compare with competitors? Is having an NA as a spam score a bad thing?
Does anyone know the factors considered in the new domain authority algorithm other than spam score and complex distributions of links based on quality and traffic? Does anyone know of on-site factors we can use to compare with competitors to try and improve DA? Is having an NA as a spam score a bad thing?
Moz Bar | | CQMarketing0 -
Spam site
my website is activate in egg incubation industry https://www.taksafir.com . Level of this site spam score is 11% . now how can i reduce that ?
Moz Bar | | HeidiMaryAyuningtyas0 -
Calling all 301 htaccess Guru's - www to non www - then to https + Redirect homepage to inner page
I have tried searching, multiple opinions and multiple things that supposedly work. What I have now, seems to work from an end user perspective, but Roger tells me otherwise: Redirect Chain issue....redirect, which redirects which redirects etc..... FIRST, we need to redirect all www to non www. SECOND, we need to redirect all to https. THIRD, we need to redirect the homepage to an inner page. (Got to love BOGUS DMCA complaints! :)?) So far we have: RewriteEngine on
Moz Bar | | Jes-Extender-Australia
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
RewriteCond %{HTTP_HOST} ^mydomain.com.au$ [OR]
RewriteCond %{HTTP_HOST} ^www.mydomain.com.au$
RewriteRule ^/?$ "https://mydomain.com.au/inner-page-here" [R=301,L] Plus down the page there is the usual wordpress settings: <ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]</ifmodule> So, why does it seem to work for the end user, but Roger has his knickers in a knot saying, redirect, to redirect to redirect etc? Namaste and many thank you's in advance 🙂0 -
I have recently switzed to SSL on ramxpert.dk, now Moz is not able to crawl the site anymore
Hi Moz I have recently switzed to SSL on ramxpert.dk, now Moz is not able to crawl anymore. When I look at the crawl report there is just one fault displayed: "403 : Received 403 (Forbidden) error response for page." How can i solve this issue asap? I was not aware that there would be any issues with moz when activating a SSL certificate on the site. The campaign with the problem is: ramxpert.dk. Regards
Moz Bar | | WebBoost
Jesper Nielsen0 -
Need to solve "Oops our crawlers were unable to access" url for new campaign
I'm putting the url designfirstkitchenandbath.com and getting the "oops! our crawlers were unable to access the site. Since this site is a potential client, which shows up online, I can't get access to fix the code, plus while I can write a little html I don't feel comfortable working with hard, live code on someonelse's site. Anyone have a simple solution?
Moz Bar | | alisacromer0 -
Why RogerBot can't crawl site https://unplag.com
Hello Please help me to solve the problem. The on-page grader and Crawl Test are not working for Unplag.com website. Both said that they can't access the url. Yes, I've tried different variants like unplag.com, http://unplag.com One more thing - RogerBot was disallowed in robots.txt file. I deleted it from the file a week ago so maybe moz index haven't been renewed.
Moz Bar | | Targeras0 -
Open site explorer linking root domains
My company has been trying to increase the number of linking root domains for a specific page on our website using our PR company and press releases that are sent out and linked back to this page. This is working nicely, but the number of linking root domains is still not increasing under the "linking root domains" tab. I am noticing the correct links to this domain under "fresh web mentions" though. I know this tool can take a bit to update, but it has been quite some time and I still only see the one link.
Moz Bar | | isret_efront0