Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Best practice to prevent pages from being indexed?

Intermediate & Advanced SEO

456

Locked

TheaterMania last edited by

Generally speaking, is it better to use robots.txt or rel=noindex to prevent duplicate pages from being indexed?
1 Reply Last reply
Reply Quote 0
Andy-Halliday last edited by

Isn't the main question: Why do you have duplicate pages, are these essentials - the easiest option would be to remove them. But in terms of whats the best option, here is a great article from Moz: http://moz.com/learn/seo/robotstxt

I would read that and decide based on your websites and situation the option best suits you.

In my opinion I would suggest:
1 Reply Last reply
Reply Quote 1

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Best SEO Practices for Displaying FAQs throughout site?

I've got an FAQ plugin (Ultimate FAQ) for a Wordpress site with tons of content (like 30 questions each with a page full, multi-paragraphs, of answers with good info -- stuff Google LOVES.) Right now, I have a main FAQ page that has 3 categories and about 10 questions under each category and each question is collapsed by default. You click an arrow to expand it to reveal the answer.I then have a single category's questions also displayed at the bottom of an appropriate related page. So the questions appear in two places on the site, always collapsed by default.Each question has a permalink that links to an individual page with only that question and answer.I know Google discounts (doesn't ignore) content that is hidden by default and requires a click (via js function) to reveal it.So what I'm wondering is if the way I have it setup is optimal for SEO? How is Google going to handle the questions being in essentially three places: it's own standalone page, in a list on a category page, and in a list on a page showing all questions for all categories. Should I make the questions not collapsed by default (which will make the master FAQ page SUPER long!)Does Google not mind the duplicate content within the site?What's the best strategy?
Intermediate & Advanced SEO | | SeoJaz

0
Why do I have so many extra indexed pages?

Stats- Webmaster Tools Indexed Pages- 96,995 Site: Search- 97,800 Pages Sitemap Submitted- 18,832 Sitemap Indexed- 9,746 I went through the search results through page 28 and every item it showed was correct. How do I figure out where these extra 80,000 items are coming from? I tried crawling the site with screaming frog awhile back but it locked because of so many urls. The site is a Magento site so there are a million urls, but I checked and all of the canonicals are setup properly. Where should I start looking?
Intermediate & Advanced SEO | | Tylerj

0
Why would my total number of indexed pages stop increasing?

I have an ecommerce marketplace that has new items added daily. In search consoloe my pages have always gone up almost every week. It hasn't increased in 5 weeks. We haven't made any changes to the site and the sitemap looks good. Any ideas on what I should look for?
Intermediate & Advanced SEO | | EcommerceSite

0
Advanced: SEO best practice for a large forum to minimise risk...?

Hi Hope someone can offer some insight here. We have a site with an active forum. The transactional side of the site is about 300 pages totals, and the forum is well over 100,000 (and growing daily) meaning the 'important' pages account for less that 0.5% of all pages on the site. Rankings are pretty good and we're ticking lots of boxes with the main site, with good natural links, logical architecture, appropriate keyword targeting. I'm worried about the following: crawl budget PR flow Panda We actively moderate the forum for spam and generally the content is good (for a forum anyway), so I'm just looking for any best practice tips for minimising risk. I've contemplated moving the forum to a subdomain so there's that separation, or even noindexing the forum completely, although it does pull in traffic. Has anyone been in a similar situation? Thanks!
Intermediate & Advanced SEO | | iProspect_Manchester

1
Removing pages from index

My client is running 4 websites on ModX CMS and using the same database for all the sites. Roger has discovered that one of the sites has 2050 302 redirects pointing to the clients other sites. The Sitemap for the site in question includes 860 pages. Google Webmaster Tools has indexed 540 pages. Roger has discovered 5200 pages and a Site: query of Google reveals 7200 pages. Diving into the SERP results many of the pages indexed are pointing to the other 3 sites. I believe there is a configuration problem with the site because the other sites when crawled do not have a huge volume of redirects. My concern is how can we remove from Google's index the 2050 pages that are redirecting to the other sites via a 302 redirect?
Intermediate & Advanced SEO | | tinbum

0
Best Way to Incorporate FAQs into Every Page - Duplicate Content?

Hi Mozzers, We want to incorporate a 'Dictionary' of terms onto quite a few pages on our site, similar to an FAQ system. The 'Dictionary' has 285 terms in it, with about 1 sentence of content for each one (approximately 5,000 words total). The content is unique to our site and not keyword stuffed, but I am unsure what Google will think about us having all this shared content on these pages. I have a few ideas about how we can build this, but my higher-ups really want the entire dictionary on every page. Thoughts? Image of what we're thinking here - http://screencast.com/t/GkhOktwC4I Thanks!
Intermediate & Advanced SEO | | Travis-W

0
Title Tag Best Practices

In light of all the Google updates in 2013, have you updated/changed your title tag best practices? Is the format of (Keyword | Brand) still working well for your optimization efforts or have you started incorporating an approach similar to this format . (Keyword in a Sentence | Brand) Thanks in advance for your opinions.
Intermediate & Advanced SEO | | SEO5Team

0
Home Page or Internal Page

I have a website that deals with personalized jewelry, and our main keyword is "Name Necklace".
3 mounth ago i added new page: http://www.onecklace.com/name-necklaces/ And from then google index only this page for my main keyword, and not our home page.
Beacuase the page is new, and we didn't have a lot of link to it, our rank is not so well. I'm considering to remove this page (301 to home page), beacause i think that if google index our home page for this keyword it will be better. I'm not sure if this is a good idea, but i know that our home page have a lot of good links and maybe our rank will be higher. Another thing, because google index this internal page for this keyword, it looks like our home page have no main keyword at all. BTW, before i add this page, google index our main page with this keyword. Please advise... U5S8gyS.png j50XHl4.png
Intermediate & Advanced SEO | | Tiedemann_Anselm

0