Crawl Test
-
Hello,
Does the Crawl Test having some issues at the moment. It seems so slow. I submitted a website to crawl test 3-4 days ago and still its in progress. This usually only takes 24hrs max. THanks.
-
Hey Heather!
Sorry about that. Can you send me an email at moz.com/help/contact and I'll have our engineers take a look!
Cheers!
-
Hi David!
I am also experiencing this issue now, and was wondering if you could help!
Thanks!
-
Anytime!
Cheers
-
Hi,
Yeah. Crawl is working fine now. THank you.
Cheers!
-
Hi Lueka
Our engineers have resolved the issue and your report is ready. Can you confirm you have received it?
Cheers!
-
Hi Lueka!
I can confirm there is an issue that we just discovered and are investigating. I will let you know when this will be fixed and I am terribly sorry for any inconvenience.
Talk to you soon!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz bot has trouble crawling Angular JS - I believe it's seeing the SPA (Single Page Application) before Universal. Anyone else have this issue? What is the fix?
The Moz bot user agent detection settings are able to read Universal, but the Single Page Application (SPA) version partially loads on the website before Universal. Because of this, Moz (and possibly search engines) think we have massive duplicate content issues. For example, the crawl report said a particular product page (which has about 1,000 words) has 33,000 words and has duplicate content with over 300 other pages. This makes me believe it's only picking up the SPA version. Has anyone come across this, and what would be the fix?
Moz Bar | | laurengdicenso1 -
Site Crawl 1-page 301 status error but httpstatus.io says its 403
I am trying to run a site crawl for my website and MOZ is only resulting in 1 page crawled with the home page URL Status Code of 301. However when I run it in httpstatus.io it is giving me a 403 status error. Im curious as to why MOZ is saying its a 301 and httpstatus.io is saying 403. Is there anything I can do in MOZ first to get the site crawled before asking my developers to look into the 403 error?
Moz Bar | | JohnConover0 -
Crawl Test is now On-Demand Crawl!
If you've been with Moz a while, you may have used our old Crawl Test tool. A year ago we launched an all new, campaign-based Site Crawl (with an entirely rebuild crawl engine), but Crawl Test fell into disrepair and we haven't had a solid tool for crawling non-campaign domains. I'm happy to announce that we've just launched an all new On-Demand Crawl, built on the new Site Crawl engine, with a UI that's focused on quick insights. Moz Pro Standard tier customers can run up to 5 crawls per month at 3,000 page per crawl (crawls are saved for 90 days), with per-month limits increasing at higher levels. Most On-Demand Crawls should run in a few minutes, making the tool perfect to get quick insights for sales meetings, vetting prospects, or analyzing competitors. We've written up a sample case study or logged-in customers can go directly to On-Demand Crawl. Try it out -- we'd love to hear your use cases (either here or in the blog post comments).
Moz Bar | | Dr-Pete6 -
Different Errors Running 2 Crawls on Effectively the Same Setup
Our developers are moving away from utilising robots.txt files due to security risks, so e have been in the process of removing them from sites. However we, and our clients still want to run Moz crawl reports as they can highlight useful information. The two sites in question sit on the same server with the same settings (in fact running on the same Magento install). We do not have a robots.txt files present (they 404), and as per Chiaryn's response here https://mza.seotoolninja.com/community/q/without-robots-txt-no-crawling this should work fine? However for www.iconiclights.co.uk we got: 902 : Network errors prevented crawler from contacting server for page. While for www.valuelights.co.uk we got: 612 : Page banned by error response for robots.txt. These crawls were both run recently, and there was no robots.txt present. Not to mention, they are on the same setup/server etc as mentioned. Now, we have just tested this, by uploading a blank robots.txt file to see if it changed anything - but we get exactly the same errors. I have had a look, but can't find anything that really matches this on here - help would really be appreciated! Thanks!
Moz Bar | | I-COM0 -
How do I cancel a crawl request?
I was farting around and exploring the Crawl Test tool, and accidentally sent out a crawl for a competitor's site (I wanted to see if the tool would decline to crawl without verification). I do NOT want to actually crawl that site, nor do I want the competitor to see that we requested it (for obvious reasons) - how do I cancel it?
Moz Bar | | mkbeesto0 -
Is there a tool that works like crawl test that allows more than 3000 pages?
I enjoy using crawl test inside moz but I need to find a way to crawl all the pages on a site. It would probably be in the neighborhood of 10,000 urls. Does anyone know of a free tool and if not is there a paid tool that will do this?
Moz Bar | | bradwayland0 -
I am not able to perform crawl test in moz tools
it is throwing there is some problem in domain when i try testing the crawl test for my domains
Moz Bar | | IBEE-Hosting0 -
Crawl Test cannot be seen on my PC. Using Windows 8.
I received and downloaded my Crawl Test. When I try to open it, my pc says "This app can't run on your PC. To find a version for your PC, check with the software publisher". I'm running Windows 8. Can I view my Crawl Test with my PC? Is there a work-around for this issue? Update I can apparently open my Crawl Test and view it as an Excel Spreadsheet. But when I download it and choose Save As, it saves it as a MS-DOS Application. This is my very first Crawl Test and I am not sure if I am doing everything right.
Moz Bar | | jameskoby010