Robots.txt Question for E-Commerce Sites
-
Hi All,
I have a couple of e-commerce clients and have a question about URLs.
When you perform a search on website all URLs contain a question mark, for example: /filter.aspx?search=blackout
I'm not sure that I want these indexed.
Could I be causing any harm/danger if I add this to the robots.txt file? /*?
Any suggestions welcome!
Gavin
-
You're right on target, it's not a good idea to index search results. Google doesn't want to crawl or index other search results in its own search results. There are some exceptions for gigantico sites like Yelp or TripAdvisor when showing their search results pages are actually the best option, but if you're not at that level and especially if you're an ecommerce site, it's not recommended.
You wouldn't be harming anything by excluding search from your robots.txt file. In fact, many top sites exclude search results to preserve crawl capacity and for indexation reasons.
You'll also want to look at parameter handling in Search Console, this article from Google will get you started.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best Site Architecture tool for analysis my & competitors site?
Hello All, I am really confused with my current architecture for Ecommerce site, can you please suggest any tool or software where I can analysis mine and competitors site architecture ? Thanks!
On-Page Optimization | | pragnesh96390 -
Time on Site vs Bounce Rate Question
If you follow my questions here on SEOMoz, you know that we have quite a few issues with a particular site. http://bit.ly/RE8V2a Our bounce rate was over 85% for quite some time. We changed the site template and the Bounce Rate has dropped to around 5%, but the Time on Site has dropped dramatically. We were just over two minutes, now we're averaging 1:34. But here's the thing, time spent on the Home Page is averaging 24 seconds! What would you take away from this? Thanks so much!
On-Page Optimization | | Linwright0 -
Google Fonts & Site Speed
Hello, Does the use of one google font slow down a website enough to effect load speed and thus rankings? Here's the ones we're choosing from: www.google.com/webfonts How do we know if the one we choose is too slow? Thank you.
On-Page Optimization | | BobGW0 -
I have a question about on page links or duplicate contant
Ok help me out here friends. I’m working with the warnings and errors for my site. I have two problems that relate to each other and I want to know if you had to choose what problem what would you choose. I’m running into some duplicate content and title errors because under categories for my products there are so many products that it creates more than one page and with each new page it has the same title or same content on the page. I tried to make this less in some cases by showing more products per page like 100 items and in most cases per category it will only show one page now. Now some times there’s still more than one page and also this creates too many links now on that category page. So I think I can get rid of all the to many on page links but it will make more pages and duplicate content and title tag. What would you guys do?
On-Page Optimization | | Dataken0 -
To Optimize Brand Name or Product Name First on Product Pages for E-Commerce Website?
We are using your free month trial for optimization of our E-Commerce website. In regards to individual product pages such as this one http://www.amgair.com/air-purifiers/iqair-healthpro-plus-air-purifier/, would it be more effective to have the page title start with the brand name and then the product (as we have it now) or forgo the brand name and start with just the product. IE: IQAir Healthpro Plus Air Purifier or HealthPro Plus Air Purifier by IQAir. These are commodity type products and are price restricted so all competitive websites advertise at the same pricing and it would be helpful not only to have a keyword phrase that is searched for a lot but also one that is easy to rank for. Please give me a recommendation when possible.
On-Page Optimization | | youhow0 -
How Can I Get Yahoo to Index My Site?
How Can I Get Yahoo to Index My Site? I have installed Bing webmaster tool two months ago -- is Yahoo that slow. My site has been out since May 2010 and for a year and a half, I only have 40 pages index. HELP!!!!
On-Page Optimization | | AppleCapitalGroup0 -
Numbers above actual site content
Most pages on my website contain many numbers above the actual text on the page. This is useful for users and looks good on an actual view of the page. However, when a bot reads the page it appears as rows of numbers with a few sentences at the bottom of the page. Does having these number have a negative SEO effect? If so, should I change them to something such as an image so they aren't readable by search engines?
On-Page Optimization | | theLotter0 -
Product Title Formating Question
The majority of the products I sell require lengthy product titles, I have 600+ items that have titles over 70+ characters. In the interest of reducing the character count (Product Feeds) I'm trying to eliminate unnecessary "stop words" in the titles, such as "and", "with" etc.. I've listed an example of a current product listing below, followed by two different formats to reduce the character count. Which one if any is better for SEO or should I leave my titles alone and stick with my original format? Current Listing Example: DeWalt Dual Comfort Safety Glasses with Black Frame and Clear Anti-Fog Lens Suggested Listing Example: DeWalt Dual Comfort Safety Glasses-Black Frame-Clear Anti-Fog Lens DeWalt Dual Comfort Safety Glasses/Black Frame/Clear Anti-Fog Lens
On-Page Optimization | | MEldridge0