How to crawl the whole domain?
-
Hi,
I have a website an e-commerce website with more than 4.600 products. I expect that Seomoz scan check all url's. I don't know why this doesn't happens.
The Campaign name is Artigos para festa and should scan the whole domain festaexpress.com. But it crels only 100 pages
I even tried to create a new campaign named Festa Express - Root Domain to check if it scans but had the same problem it crawled only 199 pages.
Hope to have a solution.
Thanks,
Eduardo -
Hi Kery, thanks, I just sent to them.
Regards,
Eduardo. -
Hi Eduardo,
I'm sorry you're still having problems. At this point, it'd be best for you to send an email to [email protected] and have our help team look at it for you. They'd be the ones who could give you the most advice for diagnosing this.
Keri
-
Still have the same problem. Isn't that an issue with SEOMoz?
The domain is www.festaexpress.com has no flash and is crawled by google with no issues.Regards,
Eduardo. -
Hi Eduardo.
The way crawlers work is the begin on your home page and "crawl". They look at all the links on your home page and follow each one to the next page, then the next until your whole site has been captured.
Why are only 100 pages being crawled?
Most likely either because your site is not very well linked, or because you don't have a good navigation system, or because your navigation and links are presented in a format such as flash which the crawler cannot read.
Another possibility would be if the crawler is being blocked or hindered by your robots.txt file.
-
Not sure, but you could try Microsoft's IIS tool to spider your site. It is possible that your site has issues that make it difficult to spider, hence why SEOMoz's bot isn't working. You could also try something like Xenu Link Sleuth or HTTrack.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My domain authority is low in MOZ and high in other apps
_I checked my site in other SEO apps and they are showing a higher DA for my site. However, in MOZ is very low. Can you please explain it to me? _what caused the difference?
Moz Pro | | K_Monestel0 -
Shopify crawl issues
Hi Moz'ers, I am a total newcomer to this level of seo. Recently I transitioned to Shopify and I'm puzzled by why I'm getting 803 errors - incomplete crawl attempts due to server timing out. Wouldn't this have to do with Shopify? How would I go about fixing it? I'm also getting 804 - SSL issues, but I assume that will go away. Any advice? Thanks! Sharon
Moz Pro | | Sharon2016
www.ZeldasSong.com0 -
Question about SEO and domains
Hello, Over time we have purchased many domain names that relate to our business. Let us say we sell chairs of type A, chairs of type B, Tables of Type C etc. We also have a main web site (main domain) which has been functioning for many years and is ranked well but not great. Since we have domain chairtypeA.com and chairtypeB.com I was thinking of putting them to good use as follows. STRATEGY A a) Create the domain as a parallel domain but is mainly informational - it talks about chair type A, its history, what to look for etc. It has the same IP address as main domain b) Have all the links on the page be directly linked to the MAIN site- so if you are on page for ChairtypeA and want to see it in a set you just click a link and it takes you to the main site. c) The look and feel of the ChairtypeA site is exactly the same as the main site - in fact except for the Heading, title and content the whole framework is from the main site. My hope is that if people are looking for chairtypeA, we would rank and attract customers. Plus it would use up some of the domain names we have purchased. THE OTHER ALTERNATIVE (Strategy B) is a) Create a domain for chairtypeA parallel to main domain (same IP address) b) redirect it to subfolder in main domain c) refer to the pages also as maindomain/chairtypeA (here is where it is no longer purely parallel) I have 3 objectives a) Use my purchased domain names b) Focus on certain products (liek ChairtypeA) c) Overall increase visibility and ranking Please give me any feedback which indicates whether Strategy A or Strategy B is more effective, or if any has negative consequences. Or you could suggest Strategy C - a completely new approach. Thanks Sam
Moz Pro | | samgold0 -
Domain or Category Links for E-Commerce Page?
I am trying to rank with my page on a major keyword for my E-Commerce store. Currently, I am ranking on page 4 in SERP:
Moz Pro | | soralsokal
mydomain.com/category-for-major-keyword Now when people like to link to me, what should I do? Shall they link to: mydomain.com or mydomain.com/category-for-major-keyword What brings me more in terms of ranking for that keyword with my category? Thanks, Robin1 -
Campaign Crawl
I have a site with 8036 pages in my sitemap index. But the MozBot only Crawled 2169 pages. It's been several months and each week it crawls roughly the same number of pages. Any idea why I'm not getting fully crawled?
Moz Pro | | JMFieldMarketing0 -
How often does seomoz crawl the site? Can you force a crawl at a specific time ?
How often does seomoz crawl the site? Can you force a crawl at a specific time ?
Moz Pro | | stewbuch18720 -
Does Open Site Explorer purposefully not crawl some sites?
I use both SEOmoz's Open Site Explorer and Web Master Tools to find backlinks when conducting link audits. WMT always finds more links than OSE; I understand Google's database is bigger. But what is interesting to me is that it seems that a large percentage of the links WMT finds that OSE does not are real crappy links that I don't want. That makes me wonder if SEOmoz decides not to crawl certain, low quality, sites? Just curious.
Moz Pro | | ILM_Marketing0 -
Is there anyway i can change a campaign setting from root domain to domain?
Is there anyway i can change a campaign setting once it has been established from root domain to domain? I could just start a new campaign but that seems a bit of a pain.
Moz Pro | | pancakes0