Page not Accesible for crawler in on-page report
-
Hi All,
We started using SEOMoz this week and ran into an issue regarding the crawler access in the on-page report module. The attached screen shot shows that the HTTP status is 200 but SEOMoz still says that the page is not accessible for crawlers. What could this be?
Page in question
http://www.tiasnimbas.edu/Executive_MBA/pgeId=307Regards,
Coen
-
Most welcome
You can install Mozbar if you are using FF or Chrome, it has a lot of useful features.
-
Ok super thank you
-
I've just checked your Google Cache for this page and it seems to be fine. I advise you don't worry that much about SEOmoz robot.
-
Hi, Yes i have check it through the webmaster tools :
`Result: HTTP/1.1 200 OK Cache-Control: public, max-age=1 Content-Type: text/html; charset=utf-8 Content-Encoding: gzip Expires: Tue, 06 Dec 2011 14:27:57 GMT Last-Modified: Tue, 06 Dec 2011 14:27:56 GMT Vary: Accept-Encoding Server: Microsoft-IIS/7.5 City: /Executive_MBA/pgeId=307 Set-Cookie: ASP.NET_SessionId=wb3k3i45cxmag3mh31swghbz; domain=.tiasnimbas.edu; path=/; HttpOnly X-AspNet-Version: 2.0.50727 X-Powered-By: ASP.NET Date: Tue, 06 Dec 2011 14:27:56 GMT Content-Length: 12991 The robot.txt allows all crawlers:` User-agent: * Sitemap: http://www.tiasnimbas.edu/sitemap.xml
-
Hi have you tried to check if this page accessible for Google robots in Webmaster Panel. Because sometimes there are some sort of software which are blocking SEOmoz robot.
Also check your robots.txt file if there are no any restrictions. For ex. it might be setup to accept only Google robot but not others.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does a no-indexed parent page impact its child pages?
If I have a page* in WordPress that is set as private and is no-indexed with Yoast, will that negatively affect the visibility of other pages that are set as children of that first page? *The context is that I want to organize some of the pages on a business's WordPress site into silos/directories. For example, if the business was a home remodeling company, it'd be convenient to keep all the pages about bathrooms, kitchens, additions, basements, etc. bundled together under a "services" parent page (/services/kitchens/, /services/bathrooms/, etc.). The thing is that the child pages will all be directly accessible from the menus, so there doesn't need to be anything on the parent /services/ page itself. Another such parent page/directory/category might be used to keep different photo gallery pages together (/galleries/kitchen-photos/, /galleries/bathroom-photos/, etc.). So again, would it be safe for pages like /services/kitchens/ and /galleries/addition-photos/ if the /services/ and /galleries/ pages (but not /galleries/* or anything like that) are no-indexed? Thanks!
Technical SEO | | BrianAlpert781 -
Duplicate Page Content
Hello, After crawling our site Moz is detecting high priority duplicate page content for our product and article listing pages, For example http://store.bmiresearch.com/bangladesh/power and http://store.bmiresearch.com/newzealand/power are being listed as duplicate pages although they have seperate URLs, page titles and H1 tags. They have the same product listed but I would have thought the differentiation in other areas would be sufficient for these to not be deemed as duplicate pages. Is it likely this issue will be impacting on our search rankings? If so are there any recommendations as to how this issue can be overcome. Thanks
Technical SEO | | carlsutherland0 -
Different meta descriptions for same page
Hi Depending what terms I put into Google I am seeing a different meta description for exactly the same page. I have checked Umbraco CMS and everything seems in working order. Is there a reason this would be happening? Anyone else had trouble like this?
Technical SEO | | TheZenAgency0 -
Problems with to many indexed pages
A client of our have not been able to rank very well the last few years. They are a big brand in our country, have more than 100+ offline stores and have plenty of inbound links. Our main issue has been that they have to many indexed pages. Before we started we they had around 750.000 pages in the Google index. After a bit of work we got it down to 400-450.000. During our latest push we used the robots meta tag with "noindex, nofollow" on all pages we wanted to get out of the index, along with canonical to correct URL - nothing was done to robots.txt to block the crawlers from entering the pages we wanted out. Our aim is to get it down to roughly 5000+ pages. They just passed 5000 products + 100 categories. I added this about 10 days ago, but nothing has happened yet. Is there anything I can to do speed up the process of getting all the pages out of index? The page is vita.no if you want to have a look!
Technical SEO | | Inevo0 -
Duplicate Home Page
Hi everyone! So, I;m using the crawl diagnostics in Moz and it's telling that I've got duplicate content for these two pages: http://www.bridgelanguages.com/
Technical SEO | | Bridge_Education_Group
http://www.bridgelanguages.com/index.php?p=3233&source=3 Would a redirect from the 2nd page to the 1st one be a solution? I'm not even sure where that 2nd link is on the site? Any suggestions or has anyone experienced the same? Thanks! Kelly0 -
Is the Authority of Individual Pages Diluted When You Add New Pages?
I was wondering if the authority of individual pages is diluted when you add new pages (in Google's view). Suppose your site had 100 pages and you added 100 new pages (without getting any new links). Would the average authority of the original pages significantly decrease and result in a drop in search traffic to the original pages? Do you worry that adding more pages will hurt pages that were previously published?
Technical SEO | | Charlessipe0 -
Are all duplicate pages bad?
I just got my first Crawl Report for my forum and it said I have almost 9,000 duplicate pages. When I looked at a sample of them though I saw that many of them were "reply" links. By this I mean the "reply" button was clicked for a topic yet since the crawler was not a member, it just brought them to the login/register screen. Since all the topics would bring you to the same login page I'm assuming it counted all these "reply" links as duplicates. Should I just ignore these or is there some way to fix it? Thanks in advance.
Technical SEO | | Xee0 -
Page MozRank and MozTrust 0 for Home Page, Makes No Sense?
Hey Mozzers! I'm a bit confused by a site that is showing a 0 for home page MozRank and MozTrust, while its subdomain and root domain metrics look decent (relatively). I am posting images of the page metrics and subdomain metrics to show the disparity: http://i.imgur.com/3i0jq.png http://i.imgur.com/ydfme.png Is it normal to see this type of disparity? The home page has very little inbound links, but the big goose egg has me wondering if there is something else going on. Has anyone else experienced this? Or, does anyone have speculation as to why a home page would have a 0 MozRank while the subdomain metrics look much better? Thanks!
Technical SEO | | ClarityVentures0