Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Translated the site but traffic is not coming
Hi, We've build a lawyer directory website (www.iranianlawyers.com) which already has good Google rankings for related terms (Ex. Iranian lawyers, Iranian lawyers california, etc). About 1.5 months ago we translated the site to Farsi and published it online: www.iranianlawyers.com/farsi However we have yet to see any new traffic generated from those pages. The website has a decent back-link profile and there are no almost no competitors in our space with translated pages. Would someone please take a look at our translated pages and let me know if there are any major on-site issues that you see that we need to address? I've checked for noindex or nofollow tags but they dont seem to be an issue. Not sure if I'm missing something here. Thank you very much
On-Page Optimization | | Heydarian0 -
Potential new URL structure for my ecommerce site
At the moment my site suffers from a flat product category structure where over 600 items fall into one category alone. This category is then filtered using a faceted search which appends query strings to the category URL and changes the products displayed on the page. At the moment our product category URL is as follows, www.domain.com/category/greeting-cards and this holds all cards including occasions such as anniversary, birthdays etc and also themes such as animal cards, contemporary cards etc I have proposed changes to my developer to change this structure to include subcategories. I can now go two subcategories deep. For example, "greeting cards > occasions > birthday cards" or "greeting cards > themes > animals". This is reflected in the new URL structure, which has been proposed, www.domain.com/greeting-cards/occasions/birthday-cards. In this URL do I need "occasions" in the URL as I don't think it adds much value to the user? Would I be better of having www.domain.com/greeting-cards/birthday-cards. If a user searches for "birthday cards" then I think this would be more relevant?
On-Page Optimization | | joe-ainswoth0 -
Canonical URL Tag Usage
I have a large website, almost 1500 pages that each market different keywords for the trucking logistics industry. I don't really understand the new Canonical URL Tag USAGE. They say to use it so the page is not a duplicate but the page that MOZ is call for to have the tag isn't a duplicate. It promotes 1 keyword that no other page directly promotes. Here is the page address, now what tag would I put up in the HEAD so google don't treat it as a duplicate page. http://www.freightetc.com/c/heavyhaul/heavyhaul.php 1. Number 1 the actual page address because I want it treated like its own page or do I have to use #2 below? 2. I don't know why I would use #2 as I want it to be its own page, and get credit and listed and ranked as its own page. Can anyone clarify this stuff to me as I guess i am just new to this whole tag usage.
On-Page Optimization | | dwebb0070 -
On-Site Optimization Issue!
Hello, I have some confusion about how to structure my site to better in on-site optimization. I am using WordPress. Therefore, there are many things that I need to consider as following: Static Page for homepage OR Latest posts? Archive, Category, Author, Attachment and Tag pages - To put meta robots (no index, follow) or not to prevent duplication? If I use Static Page for homepage, do I need to add meta robots (no index, follow) to POINT 2 above or not? If I use Latest Posts for homepage, do I need to add meta robots (no index, follow) to POINT 2 above or not? To have breadcrumb or not? To have recent posts, comment, tag clouds or popular posts/comments widget or not? To have social sharing icons and related posts in single post or not? If you don't mind adding more tips that I don't know it would be very great! Thanks!
On-Page Optimization | | dinabrokoth0 -
Why is my site not ranking?
Could you please help me understand what is wrong with this site: www.award-certificates.com It simply isn't ranking after about 3 years and I am not so sure what I can do to improve it.
On-Page Optimization | | nicolebd0 -
Content for ecommerce site
How important on site/page contents are for ecommerce site. Keeping in mind the page layout. Its not that important to have page copy/content at all for ecommerce sites If yes, does position of content is an important factor? if putting page copy/content in upper fold of a page then the most important thing which is product itself will have less exposure if putting near the footer of the page, does that seem like doing just for the sake of SEs and ranking. How important internal linking form that content would be compare to left panel links or links at the header of a website Thanks Rick
On-Page Optimization | | RickGa0 -
How many urls per page is to many
I know it used to be 100 urls per page, but recently Matt cutts has said that they can count a lot more now. I was wonder what you guys thought was how many was to many per page?
On-Page Optimization | | Gordian0