My page cant be crawled by MOZ
-
Hi everyone,
I have a page named zita.vn. Im using Moz to do SEO for the page, but when I use On-page grader in MOZ, I got an error that the page cant be accessed. Additionally, I got an issue with page crawling.
Google search still crawl my page normally.
Please help.
Thanks a lot.
-
Hi there,
I have been checking my site for a week, but I havent seen any new updates about my site in backlink. Im sure that since last week there are a few sites pointing a link to my website but I havent found it on Moz.
Please explain it for me.
Tks a lot,
-
The useragent is: rogerbot
-
Hi Kristina,
Thank you for your answer.
Currently to protect our site from being hacked we need your user agent to add it into our whitelisted, so you will be able to crawl our website.
Regards,
Phuong.
-
Hi there! Kristina from Moz's Help Team here.
When our crawler attempts to access your site, we are running into this error: http://screencast.com/t/CmprKKb7a "403 Forbidden".
This normally means that you're blocking us on a server level. When I ran your site in a header checker I was able to see a 200, but when I curled your site from our servers it returned a 403.
The bummer here is that I can't do anything on the support side to fix this.
This 403 error is coming directly from your server, so at this point the best thing to do will be to send this information to your web dev. They'll need to make sure our crawler (rogerbot) is whitelisted, so our tools can start working hard for you.
As always, you can contact our team on issues like this in the future by emailing [email protected], but hopefully this helps point you in the right direction towards a solution here.
Thank you,
-Kristina -
I'm seeing a 403 response for your website:
wget zita.vn
--2016-08-02 11:46:09-- http://zita.vn/
Resolving zita.vn... 107.167.188.173
Connecting to zita.vn|107.167.188.173|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.zita.vn/ [following]
--2016-08-02 11:46:09-- https://www.zita.vn/
Resolving www.zita.vn... 107.167.188.173
Connecting to www.zita.vn|107.167.188.173|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-08-02 11:46:10 ERROR 403: Forbidden.If Moz is seeing a 403 they're not going to be able to crawl.
-
there is a possiblity of this issue is due to TLD, as i believe Moz crawler do not support to all TLDs as of yet. this could be the only reason comes to my mind. However, i reckon to raise a suppert ticket for more certian answer for this.
-
Maybe you've set a number of threads too high and pause between requests too low? Some hosts will block IPs that send too many requests. How much is too many depends on your host.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz claims we have meta noindex but we don't
Hi, I'm encountering an issue where moz scan says we have meta noindex, but I have confirmed across several of our pages that this simply isn't true. I have confirmation that the below tag is present: name="robots" content="index, follow" /> I also verified our https header through https://www.webconfs.com/http-header-check.php and see nothing indicating that we are sending any no index headers. Why would the crawler report this when it doesn't seem to be the case? Let me know if I need to provide more information.
Moz Bar | | charper_floqast0 -
Duplicate Content & Title Tag Group Fields on MoZ Report
Hello, On my SEO MOZ exported Site Crawl CSV report, I have columns for Duplicate Content Group & for Duplicate Title Tag Group. The values in the columns are numerical - 20, 5 , 15, etc. Can anyone explain to me what these values represent and how I can fix the issues I presume they represent? Thank you,
Moz Bar | | AED-1
Scott0 -
Site Crawl 1-page 301 status error but httpstatus.io says its 403
I am trying to run a site crawl for my website and MOZ is only resulting in 1 page crawled with the home page URL Status Code of 301. However when I run it in httpstatus.io it is giving me a 403 status error. Im curious as to why MOZ is saying its a 301 and httpstatus.io is saying 403. Is there anything I can do in MOZ first to get the site crawled before asking my developers to look into the 403 error?
Moz Bar | | JohnConover0 -
Error in Duplicate Content Being Reported - Pages Aren't Actually Duplicates
The recent crawl of one of our sites revealed a high number of duplicate content issues. However, when I viewed the report for pages with duplicate content I noticed almost all of them are not duplicates. For example, these two pages are marked as dupes:
Moz Bar | | M_D_Golden_Peak
https://www.writersstore.com/publishers/hollywood-creative-directory
https://www.writersstore.com/authors/g-miki-hayden These are thin as far as content goes but definitely not duplicates. Any recommendations or ways to adjust the settings so that these false positives aren't clogging up our site crawl report?0 -
Moz Page Analysis Country different to Who.is?
If I analyse a domain with Moz Page Analysis tool, it says that the domain is hosted in the United States but if look up the same domain on who.is, the hosting location is Italy?
Moz Bar | | Marketing_Today0 -
On-page Grader API?
Hi guys, I have a massive list of keywords i want to optimise against a specific URL. I'm checking them manually with the MOZ on-page grader to determine if the keyword is optimised for the URL (Grade A score). I was wondering if there is a way to pull the scores automatically for each URL with the MOZ api? Or does anyone know of any tools which can automate this process? Cheers, Chris
Moz Bar | | jayoliverwright1 -
Rank Tracker page shows only first 50 entries
On the Rank Tracker tool page, http://moz.com/researchtools/rank-tracker, when I hit the Next 25 it works the first time but none of the others. Trying to run a report and since we have more than 50 keywords I have to manually click on them to be checked.
Moz Bar | | HopeIndu0