Search directory - How to apply robots
-
Hi.
On the site I'm working on, we use a search directory to display our search results. It displays as follows -
With the dynamic search results appearing after the hash tag.
Because of the structure of the website, many of the lefthand nav defers back to this directory. I know that most websites "noindex, nofollow" the search results pages, but due to the ease of customers generating them, I'm afraid that if I do this, we'll miss out on the inevitable links customers will provide...and, even though it's just the main search directory, these links will still help my domain. The search is all java-generated so there's nothing for spiders to follow within this directory - save the standard category nav.
How should I handle this?
Thanks.
-
That was my first instinct as well - but I wanted to check if there was a better move? Thanks for your help!
-
Have you considered switching to simply "noindex, follow" for these pages? This would both keep them out of the search index and would allow for the benefit of external links to these pages.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How does ARIA-hidden text appear to search engines
I'm having trouble getting my accessibility team to add alt text to our site's images for SEO benefits as they feel some of it would add additional noise for screen readers. They proposed using ARIA-hidden attributes to hide the text but I'm wondering if will that be interpreted as a cloaking tactic to search engines? Also, I'm wondering if it the alt text will carry the same weight if ARIA-hidden is used. Has anyone had any experience with this? I'm having trouble finding any research on the topic.
Web Design | | KJ6001 -
Website Form Doesn't Work for Visitors from Google Search App
I've got a strange issue where if visitor is viewing a website from within the Google Search App, the app doesn't seem to be able to handle online forms. The form relies users on entering data, which is then POSTed to an external booking engine site. Normally this works fine, except when Google Search App is acting as the browser, the post payload is empty and the URL breaks. We're a bit stumped as to how to move forward. So far the only lead is that Android users can override having the app behave as a browser - but doesn't seem like iOS users can. Any additional ideas/tips are welcome here - thanks. 🙂
Web Design | | mirabile0 -
Hi, I have a doubt. If we want to hide unwanted text in a web page its possible with "" tag. And my question "does a search engine crawl those text? help me.
I want to hide a lot of text behind my site page. I know its possible with that tag. But in what way a search engine looks at those text? Hidden or they are crawled and indexed.
Web Design | | FhyzicsBCPL0 -
How do we get search engine bots to the item detail pages?
The problem we have is that we have lots of inventory pages. These inventory pages have a bunch of links at the top linking to different styles of the item, up to 56 links in some cases. Then each item listed has a link to the item's detail page and a link to the item's shop owner's page. So if a page has 50 items shown, there are really 100 links just for the inventory. This is not taking into account the header links, footer links, sidebar links to other sections on the site. We have all these links to help consumers move through the site. The problem is that every item detail page on the site is not getting indexed and actually I think it's more like over 50% of the item detail pages are not indexed because the search engines are too busy following all these other links. Should we nofollow, index the links to the different styles of the item, the shop owner page? Or what should we do to get the search engine bots to index our item detail pages?
Web Design | | CFSSEO0 -
Main page redirect affecting search results?
Question.... A recent change was made to our page www.BGU.edu by a marketing person. So now when you type in www.BGU.edu it actually redirects to a different page www.BGU.edu/inquiry This is a really bad idea isn't it? I do not know enough about SEO to know a lot, and just joined SEOmoz but do I need to tell the admin to change it back?
Web Design | | nongard10 -
Robots.txt - Allow and Disallow. Can they be the same?
Hi All, I need some help on the following: Are the following commands the same? User-agent: * Disallow: or User-agent: * Allow: / I'm a bit confused. I take it that the first one allows all the bots but the second one blocks all the bots. Is that correct? Many thanks, Aidan
Web Design | | Presenter0 -
Looking for a Static Blog Roll (Directory) plug in / tool
I am looking to add a static page to my site to allow people to add their site to a 'master directory". I'd like to make this as SEO friendly as possible. Here's it what I would really like to be able to do. 1. Someone visits the page and can either browse the blog directory on that page, or 2. Fill out a short from that has their blog name, url, short description of their blog, their state, country, and maybe the option to upload an image as well. This way people can search by the state / country to find people near by. 3. I'd like the links to be no-follow. 4. I'm not sure if there is an easy way to do this (I think I've seen some plug in's that may be able to do it.) 5. I'm also not sure if there is anything I'm missing here that could make this better. What I don't want is just a long list of links like everyone has on their side bar. I'd like to make it better, more user friendly, more seo friendly, and more helpful than that. Any creative input on how I might do this is appreciated. Thanks. 🙂
Web Design | | NoahsDad0 -
Any reaction to the announcement from Google that 'signed in' searches won't pass through search query info to analytics?
Seems like SEO is about to get that much harder: http://analytics.blogspot.com/2011/10/making-search-more-secure-accessing.html Any thoughts on this?
Web Design | | PaulM011