Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Company name in Site Meta description?
Ok. So I know that you should have your company name in the site title, but is it all that important in the site description? The reason I ask is because I am competing with another company for the #1 position (I was number 1, now #2) that has an 8 character name and mine is 22 taking away from a great deal of real estate in my 150-160 character site description in which I could provide additional information describing my company. Should I remove my name in the site description enabling me to use more descriptive keywords and actionable text such as (Find, research, contact, professional, info) etc. Thanks,
On-Page Optimization | | photoseo10 -
Does CloudFlare Benefit my site?
I am using the $20 / month CloudFlare service. My site is not running faster, so my question is: is this service adding real value? I do not have a lot of highly sensitive customer data (some email addresses and customer names and nothing more - no credit cards). I am already using Amazon Cloud Server and it appears to work fine. Any thoughts appreciated.
On-Page Optimization | | khi50 -
URL Keyword Variations?
I'm aware that keywords in the url aren't as effective as they used to be, but I'm still convinced that they do have a significant impact (based on results in one of the niches I'm in). My question is, will variations of keywords and "hidden" keywords have as much value as an exact keyword? For example, let's say that I'm trying to target the keyword "day." Will including variations like "daily" in the url work just as well? What about a brand name that includes the keyword hidden in its name, like "Dayest"? And, as a followup question, does including "stop" words have any effect? For example, if I'm trying to target the keyword "Day of the Month", would including "day" and "month" in the url be just as effective as including "day of the month"?
On-Page Optimization | | JABacchetta0 -
How to Recover From Unstable Site Structure History
I have a site that has suffered several phases of restructuring. Apparently its owners were unsure as to which direction to take when it came to structuring their content and URL schema and subjected the site to several rounds of poorly thought through implementations (i.e. example.com/content/page-title, example.com/page-title, example.com/"silo"/page-title, etc.), all within a 8 month period. I posted the originating question here on this Q&A Forum. I want to thank EGOL and Cody for taking a stab at it. What would be a good strategy to help a site like the one I describe above begin ranking again?
On-Page Optimization | | UplinkSpyder0 -
Dates in URL's
I have an issue of duplicate content errors and duplicate page titles which is penalising my site. This has arisen because a number of URLs are suffixed by date(s) and have been spidered . In principle I do not want any url with a suffixed date to be spidered. Eg:- www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/06_07_13/13_07_13 http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/20_07_13/27_07_13 Only this URL should be spidered:- http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm I have over 10,000 of these duplicates and firstly wish to remove them on block from Google ( not one by one ) and secondly wish to amend my robots.txt file so the URL's are not spidered. I do not know the format for either. Can anyone help please.
On-Page Optimization | | carbisbayhols0 -
Checking for content originality in a site
two part question on original content How would you go about checking if a site holds original content accept the long search quary within Google? ans also if I find many sites carrying my content and I am the original source should I replace the content? thanks
On-Page Optimization | | ciznerguy0 -
Nofollow on site-wide banner links
Hi I have a blog in relation to my webshop. On the blog I have two banners linking directly to the webshop on every page (site-wide links). Should I make the banner links nofollow? Thanks, Rasmus
On-Page Optimization | | secubi0 -
Linking within Secondary Site
So we've got a secondary site that has quite a bit of authority & links that used to have all types of info on parasailing. All those pages are gone and homepage is now a salespage (management decision, not mine) Our main site sells a wide range of tours and activities and does have a page for parasailing. The secondary site uses the same template/navigation as our main site (again, not my decision). Do you think that's an effective way to send link juice to our main site? The secondary site has some pretty awesome high authority sites linking to it. I've considered 301'ing the whole site to our main site but it's got a really solid domain name and I'd like to take up 2 SERP listings (main and secondary site) Is there a better way to have double listings but still send a good amount of link juice?
On-Page Optimization | | SoulSurfer80