Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On-Page Optimization on a service based Site?
Respected members, my question is that if I Wanna run a website that provide service like olansi do, But i need to know the On -page effect like How Can I make a plan to secure my position against my Service based competitor site?
On-Page Optimization | | younus_7831 -
What word should I use in my URL for my blog
Should I use the word "blog" in my sub folder as in : http://www.mybusiness.com/blog or should I use http://www.mybusiness.com/news. Is there a difference for when my site is crawled. I understand that a blog works a little differently. Can someone explain the basics?
On-Page Optimization | | graemesanderson0 -
Any scripts for automated interlinking of sites?
I have heard about similar plugins for Wordpress, but I need something like this to run on all kind of sites, no matter the CSM. Are there universal scripts capable of doing automatic interlinking of pages to rise their weight for SEO purposes? Could you share links to such scripts/sites?
On-Page Optimization | | poiseo0 -
Increasing pageviews on a social networking site
Our social networking site, Yookos, has a target of 25 million pageviews by yearend 2013. This is part of my responsibilities. As with all social platforms, user authentication is required before one can access the content. What can i do to improve the pageviews exponentially?
On-Page Optimization | | seoworx1231 -
What are the benefits of the URL meta tag?
We have too many meta tags and want to get rid of all the outdated ones. However, we don't want to eliminate valuable meta tags by mistake. So, before we say goodbye to the URL meta tag, we want to make sure we understand the pros and cons, if any. By the way, we are not referring to canonical URL tags, just URL as in:
On-Page Optimization | | GRIP-SEO0 -
How is my on-site SEO looking like?
I know this is a broad question. My site's content has been written more than one year ago and haven't been changed so far. Our main goal is to make the application hosted in the site work better every day, so we don't worry much about writing content. The URL is http://www.onlinelogomaker.com
On-Page Optimization | | rpedri0 -
Number of characters to duplicate content
I wonder how much characters in a page title so it can be characterized for Googleas duplicate content?
On-Page Optimization | | imoveiscamposdojordao
Sorry for the English, I used Google Translator.
I'm from Brazil 😄
Thanks.0 -
URL best practices, use folders or not ?
Hi I have a question about URLs. Client have all URL written after domain and have only one / slash in all URLs. Is this best practice or i need to use categories,folders? Thanks
On-Page Optimization | | 77Agency0