SeoMoz robot is not able to crawl my website.
-
Hi,
SeoMoz robot crawls only two web pages of my website. I contacts seomoz team and they told me that the problem is because of Javascript use. What is the solution to this? Should I contact my webdesign company and ask them to remove Javascript code?
-
Hi Maria,
Your menu is not optimized, but the real problem is not coming from there. Your home http://www.medixschool.ca/ has a meta refresh leading to http://www.medixschool.ca/Home/index.php instead of a 301 redirect.
You should remove this meta refresh and replace it by a 301 redirection.
Best regards,
Guillaume Voyer. -
Thanks Maximise,
I am in contact with my web developing company and looking forward for a response. Mean while it will be great if you can let me know more about it and how to go ahead with this.
thanks
-
Hi Maria,
I've just had a very brief look and it seems that the menu is quite reliant on JavaScript (try disabling Javascript and you will see that you can't access most of your pages). Crawlers do not use JavaScript so this could be a problem.
The ideal way to do this sort of thing is by showing and hiding menus with CSS, not JavaScript. I can have a better look and provide you with more info tomorrow if you need it.
-
Hi,
Thanks for the quick response. My website address is
http://www.medixschool.ca/Home/index.php
I checked the code and I can see the links.
-
You'll need to post the URL of your website for us to give you a definite answer. Try browsing to your website then right click and select 'view source'. if you can't see the links in the source code then crawlers can't see them either.
Sometimes menus are built badly with JavaScript, you will probably need to ask your developers to change your menu.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Value of having a good crawl budget?
Hi, I've seen several questions where people give advice on how to increase the crawl budget. What I haven't seen anyone comment is what the value of this really is if you have many pages that doesn't get updated very often. Take for example the typical agency page - 50 pages, most of them rarely gets updates. In a monthly basis normally 10% of the website gets updated. Is there really any value then of having 100% of the website crawled on a daily basis?
Technical SEO | | Inevo0 -
Robots.txt
Hello, My client has a robots.txt file which says this: User-agent: * Crawl-delay: 2 I put it through a robots checker which said that it must have a **disallow command**. So should it say this: User-agent: * Disallow: crawl-delay: 2 What effect (if any) would not having a disallow command make? Thanks
Technical SEO | | AL123al0 -
My beta site (beta.website.com) has been inadvertently indexed. Its cached pages are taking traffic away from our real website (website.com). Should I just "NO INDEX" the entire beta site and if so, what's the best way to do this? Please advise.
My beta site (beta.website.com) has been inadvertently indexed. Its cached pages are taking traffic away from our real website (website.com). Should I just "NO INDEX" the entire beta site and if so, what's the best way to do this? Are there any other precautions I should be taking? Please advise.
Technical SEO | | BVREID0 -
CDN Being Crawled and Indexed by Google
I'm doing a SEO site audit, and I've discovered that the site uses a Content Delivery Network (CDN) that's being crawled and indexed by Google. There are two sub-domains from the CDN that are being crawled and indexed. A small number of organic search visitors have come through these two sub domains. So the CDN based content is out-ranking the root domain, in a small number of cases. It's a huge duplicate content issue (tens of thousands of URLs being crawled) - what's the best way to prevent the crawling and indexing of a CDN like this? Exclude via robots.txt? Additionally, the use of relative canonical tags (instead of absolute) appear to be contributing to this problem as well. As I understand it, these canonical tags are telling the SEs that each sub domain is the "home" of the content/URL. Thanks! Scott
Technical SEO | | Scott-Thomas0 -
Help Website Plumetting :(
Hi I have been smacked by the penguin/panda and traffic plumetted back in April/May. We are still trying to recover and am looking at all of the potential issues. I have since cleaned up the site as much as i can and attempted to remove as much duplicate content as possible which is automatically generated by Zencart. We add content regularly and have new product reviews everyday and all product page are kept fresh as they show the last 12 customers engraving details which change daily on popular items. Could someone give me some pointers as i am hitting my head against the wall and only seeing traffic drop all the time, it's soul destroying just how much work i am putting into this every day without any effect. Site is www.keepitpersonal.co.uk Kind Regards Andy
Technical SEO | | SmithyWhiffy0 -
Invisible robots.txt?
So here's a weird one... Client comes to me for some simple changes, turns out there are some major issues with the site, one of which is that none of the correct content pages are showing up in Google, just ancillary (outdated) ones. Looks like an issue because even the main homepage isn't showing up with a "site:domain.com" So, I add to Webmaster Tools and, after an hour or so, I get the red bar of doom, "robots.txt is blocking important pages." I check it out in Webmasters and, sure enough, it's a "User agent: * Disallow /" ACK! But wait... there's no robots.txt to be found on the server. I can go to domain.com/robots.txt and see it but nothing via FTP. I upload a new one and, thankfully, that is now showing but I've never seen that before. Question is: can a robots.txt file be stored in a way that can't be seen? Thanks!
Technical SEO | | joshcanhelp0 -
Crawl diagnostic summary
In my crawl diagnostic summary its showing an error with duplicate page title and duplicate page content...why its been shown and how it can be rectified? I have pne page web site so i was unable to give options for sub domain name is it because of tht?I hope this error wont hamper my SEO process.
Technical SEO | | strasshgoa0 -
Website Ranking Issue
Hi, We have been performing our own onsite of offsite SEO along with external assistance and have ranked well over the years with minimal impact from Google updates. Howevr the last so called Panda update has affected us heavily pushing our main phrase 'web design melbourne' from 2nd to 7th where we have been for almost 2 months now on Google.com.au irrespective of onsite or offsite work. We have been trying to find signs of any onsite, IP, duplicate content, titles or other issues that may be holding us back to no avail. The only flag that Google webmaster tools is showing is a number of bad internal site links, which I think is a glitch with the CMS we are using. Even the SEO MOZ tool gives us a higher ranking compared to most competitors on page 1 of Google.com.au for our main phrase. The biggest difference between us and competitors is we chose to target an internal page specific to the topic rather than our homepage. With this sadi we have also reduced our keyword density and content quantity inline with the other sites homepages. Can anyone help shed some light on this? and perhaps something obvious that we have missed, or where we should be looking? Thanks.
Technical SEO | | paulsid0