Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I remove a high traffic page on my website?
For the last few years, a particular blog post on my site has gotten 3 times as much traffic than any other page, even the home page; however, the topic of the post is only moderately related to the website topic and I'm wondering if all that unrelated traffic is negatively effecting SEO for our primary keywords. Here's an example.... Site topic: Yoga retreats in Costa Rica (we want to attract people who are interested in booking a yoga retreat) Blog Topic: How to extend your visa in Costa Rica (it's related only because it's about Costa Rica and travel, and may help our visitors stay longer) Other Notes: In 4 years, visitors to that blog post have never converted. Blog post bounce rate is 56%, significantly higher than almost any other page Lots of comments on the blog post so visitors to it are engaged and find it very useful To get an accurate reading of interested visitors to the site, i always have to filter entrance visits to this post in my analytics because these users are not an accurate representation of the visitors we're trying to draw. My question: Because I get so much traffic from the blog post, which is about the visa renewal process, will Google consider the website less about yoga and more about visas? If so, will it make it more difficult to rank well for yoga in Costa Rica? Does Google say to itself, "Hey, this website can't be an authority about both yoga and visas in Costa Rica so we're going to consider it a visa site because of all the visits and engagement it gets for that topic." So should I remove the post or just leave it alone? It offers a lot of people valuable information so I would never delete it entirely, but would redirect it somewhere else. Thanks!
On-Page Optimization | | Cabaretti0 -
On Site Question: Duplicate H2...
Hi All A few on-site audit tools pull information on duplicate H2 tags on pages. This implies it's a bad thing and should be fixed - is that the case? On one of my sites the tag-line is in H2 in the header, so appears on every page... Just wondering if this is something worth fixing. Thanks
On-Page Optimization | | GTAMP0 -
Positioning one site in two languages
Hi all, I have used the plugin wpml in order to have one site in different languages (Spanish and English). The problem is that this plugin doesn't work with all in one SEO plugin so that the spanish home and the english home have the same title. Google has positionated in the first position the english one in google.es. How can I revert the situation if I fix the problem with other plugin? Thanks.
On-Page Optimization | | juanmiguelcr0 -
Dates in URL's
I have an issue of duplicate content errors and duplicate page titles which is penalising my site. This has arisen because a number of URLs are suffixed by date(s) and have been spidered . In principle I do not want any url with a suffixed date to be spidered. Eg:- www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/06_07_13/13_07_13 http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/20_07_13/27_07_13 Only this URL should be spidered:- http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm I have over 10,000 of these duplicates and firstly wish to remove them on block from Google ( not one by one ) and secondly wish to amend my robots.txt file so the URL's are not spidered. I do not know the format for either. Can anyone help please.
On-Page Optimization | | carbisbayhols0 -
Mentioning own site and keywords on here?
I have noticed that sometimes posters will talk about a site without mentioning what it is. I assume this is because it one of their clients so there is confidentiality, is there any other reason I should be aware of? its just that as I am new I am usually cautious and am considering posting my own site and mentioning all my keywords to ask for people’s verdict for my on-page SEO. Still working on it, will be ready soon, thought I would ask in advance. Regards,
On-Page Optimization | | Zoolander0 -
Canonical Tag for Ecommerce Site
My client has an ecommerce site with over 1,000 products. We have a ton of duplicates because of how their ecommerce system handles product pages. Each time a new product is added, there is a default product page created (/product/12345-product-name.aspx). Each time that product is added to a specific product category, another, separate URL is created (/product/office-chairs/12345-product-name.aspx). The site has over 1,000 duplicates (at least one for each product) because of how the ecommerce system structures URLs. We are unable to have unique content on /products/12345-product-name.aspx and /product/office-chairs/12345-product-name.aspx because both pages pull from the same database. Their webteam informed me that they can't implement canonical tags on individual pages, they must be dynamically added to the site all at once. Thus forcing me to choose all of the default product pages as primary URLs. Both types of URLs are getting indexed and the product URLs that were added to the categories are SEO friendly so I'm leary to eliminate one or the other with a canonical tag or a no index. Suggestions?
On-Page Optimization | | DynoSaur0 -
Redirecting to a keyword-rich domain URL
It's best practice to choose a domain that has keyword in it. But if someone has just launched a website and the domain name does not have keyword, is it better to purchase a new domain name that has a keyword in the name and redirect existing domain to the new domain? Will that help SEO? (This just launched website does not have any traffic or links yet.)
On-Page Optimization | | Amjath0