Moz not able to crawl our site - any advice?
-
When I try and crawl our site through Moz it gives this message:
Moz was unable to crawl your site on Aug 7, 2019. Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. Update these tags to allow your page and the rest of your site to be crawled. If this error is found on any page on your site, it prevents our crawler (and some search engines) from crawling the rest of your site. Typically errors like this should be investigated and fixed by the site webmaster.
I have been through all the help and doesn't seem to be any issues. You can check the site and robots.txt here: https://myfamilyclub.co.uk/robots.txt.
Anyone got any advice on where I could go to get this sorted?
-
Hello, have the same issue and MOZ getting "Bummer no data found" for domain https://wherebuycoin.com/ . The SEMRush and Ahrefs see this site, but MOZ not, even after 2 years. What can be wrong? Where to read why MOZ not crawling my site?
Thanks!
-
Check your website's robots.txt file to ensure that it's not blocking Moz's user-agent or specific directories/pages that you want to be crawled. Update the robots.txt file if necessary to allow Moz to crawl your site.
-
Hey,
We crawled the site just fine with Screaming Frog. Perhaps this is a Moz issue?
Looks like they are well aware of it though.
Thanks!
-
Hi,
I'm sorry to hear about this! Would you be able to write into [email protected] as this will allows us to investigate into this via the most convenient method.
Looking forward to hearing from you,
Eli
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
MOZ flagging urls as longer than 75 characters when they aren't
As an example: https://www.abthermal.com/3m-70-70hdt-self-fusing-silicone-rubber-electrical-tape.html Moz has flagged this as URL too long... but guideline is that happens at 75 characters or more? I count 55..?? or does the 75 limit have to include the domain? Thanks, Adrian
Getting Started | | abtechgroup0 -
Moz not recognising url to setup a new account
I am attempting to setup a new account for my client but Moz is having a problem and not registering the url http://www.esc.uk.net/. I get the "oops" message & read the info on the FAQ page it refers to. Why do I keep getting an "Oops" message when I type and submit my campaign URL? To create a campaign, we need to receive a valid HTTP response from the web server hosting your site. While the site may be accessible from a standard web browser, other user-agents and search engine bots may receive a different response. If you receive this error, we recommend running a test by sending an HTTP response using any online HTTP tool. Using an online HTTP tool I believe I received an HTTP response. What else can I do? Regards Terry
Getting Started | | Buzzin0 -
Why is Moz.com saying that none are linking to www.oneworldcetner.eu
It still says 0 after 6 weeks if i use other tooks like http://openlinkprofiler.org/r/oneworldcenter.eu#.VhbF7ivzJ8E I do see the backlinks that i was counting on was there
Getting Started | | onewordcenter0 -
Can I use 2 Moz Campaigns to Track more than 3 Competitors?
I would like to track 6 competitors for a new client, maybe short term, maybe long term. Can I set up 2 Moz Campaigns for the same client using the same keywords only with different competitors?
Getting Started | | cindyt-170380 -
My website does not allow all crawler to crawl, Now my question is that whether i need to give permission to moz crawler if yes then whaat is moz bot name?
My website does not permit all crawler to crawl website. Whether ii need to give permission to moz bot to crawl website or not? If yes what is the moz bot name?
Getting Started | | irteam0 -
Can not create new campain with my site: edunet.com.vn
I can not create new campain because it always warning my site is not a right URL. I don't understand, please tell what should I do. My site is: edunet.com.vn. (When I try to use "Grade a page for keyword" for URL: edunet.com.vn or edunet.com.vn/thong-tin-tuyen-sinh, it returns "Sorry, but that URL is inaccessible".) Thank you so much. Minh Tam.
Getting Started | | toosol0 -
Can I upgrade my moz subscription from large to premium without getting my campaigns being deleted?Do i need to wait till the end of the month to resubscribe?
I'm planning to take a large subscription of moz..but i may need a premium account in the future.Can I upgrade it in the middle of my current subscription or will i have to wait till my subscription gets over? Also, I'm planning to take only one month currently but i'll renew it once the month completes so can i continue the same account without my campaigns being deleted?
Getting Started | | kdcdmp0 -
Whenever I try to access campaigns in moz pro I get an error page
I recently signed-up for a new pro account. As I was adding my first subdomain everything was fine until I was asked to link to GA, when I clicked yes I got this error message: 403 Forbidden Now every time I click on set-up campaign I get taken to a page with nothing but the 403 Forbidden text.
Getting Started | | Toptal0