Category Pages For Distributing Authority But Not Creating Duplicate Content
-
I read this interesting moz guide: http://moz.com/learn/seo/robotstxt, which I think answered my question but I just want to make sure.
I take it to mean that if I have category pages with nothing but duplicate content (lists of other pages (h1 title/on-page description and links to same) and that I still want the category pages to distribute their link authority to the individual pages, then I should leave the category pages in the site map and meta noindex them, rather than robots.txt them. Is that correct?
Again, don't want the category pages to index or have a duplicate content issue, but do want the category pages to be crawled enough to distribute their link authority to individual pages.
Given the scope of the site (thousands of pages and hundreds of categories), I just want to make sure I have that right. Up until my recent efforts on this, some of the category pages have been robot.txt'd out and still in the site map, while others (with different url structure) have been in the sitemap, but not robots.txt'd out.
Thanks! Best.. Mike
-
Thanks, Jane! I really appreciate it.
If the now noindexed category pages have already been indexed, do you think I should request removal from the index as well?
Best... Mike
-
"I still want the category pages to distribute their link authority to the individual pages, then I should leave the category pages in the site map and meta noindex them, rather than robots.txt them. Is that correct?"
This will achieve the goal, yes. You would ideally include noindex, follow (as opposed to nofollow) in the meta tag of the page you want to exclude. This means that Google crawls the page in full and allows PageRank to flow from that page to the pages it links to, but doesn't include any of the page's content or its URL in the index.
If you exclude the page via robots.txt, Google never crawls the page at all. You sometimes see URLs whose pages have been excluded via robots.txt showing up in Google's index, because robots.txt doesn't say "don't index this URL"; it simply says "don't crawl it." That's also why excluding a page in robots.txt and putting a noindex meta tag on the page would be redundant - Google would never see the noindex tag because it would never crawl the page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help... To Optimize Category Page or Not?
My question is about whether to optimize a category page or not, but it’s a rather odd situation. Here’s a bit of background to start. When we relaunched our site, about six months ago, we had primary, secondary, and tertiary categories created. A user could reach all three levels by clicking through the site. Then we decided instead of linking to the tertiary categories, that we’d turn them into filters which can be applied at the secondary level. Thus, there is not actually a direct link to the 3rd level categories on the site. An important side note, I did check and confirm they are still included in the sitemap file. My initial thoughts were to forget any further optimization of those 3rd level categories, but as it turns out we still have rankings for some of them. Now the question… Because some of these pages are ranking and are found in the sitemap, should I include them in my SEO plan to build up and optimize, or because they are no longer linked to directly on the site will they eventually fizzle out (and I shouldn’t do anything further). This is such a unique situation that I am really looking for some insight from the community. Thanks!
Intermediate & Advanced SEO | | M_D_Golden_Peak0 -
My blog is indexing only the archive and category pages
Hi there MOZ community. I am new to the QandA and have a question. I have a blog Its been live for months - but I can not get the posts to rank in the serps. Oddly only the categories rank. The posts are crawled it seems - but seen as less important for a reason I don't understand. Can anyone here help with this? See here for what i mean. I have had several wp sites rank well in the serps - and the posts do much better. Than the categories or archives - super odd. Thanks to all for help!
Intermediate & Advanced SEO | | walletapp0 -
Ranking for homepage & category page?
We lost our Google organic ranking (position 1 - 3) for our highest converting key phrase (cotton tees) in February. The ranking was for our homepage (brandname.com) which is very image heavy and doesn't have much readable content. We noticed that all of our competitors are ranking above us for their category page, not their homepage. The difference between us and our competitors is that we specialize in this key phrase and they just offer one category of the key phrase. For example, we only sell cotton tee's and they sell cotton tees, handbags and shoes. When we dropped we noticed that Google began showing our homepage AND category page in the results, so we pointed our brandname.com to brandname.com/cotton-tees canonically. The idea was that this would assure that the homepage and category page were not competing with each other. The homepage was not really optimized for cotton tees so we thought this might help. 1. Is there any harm in removing the canonical and allowing both pages to rank? (We're also working on redesigning the homepage to add more readable text & optimize for cotton tees.) 2. Our homepage URL used to be "brandname.com/cotton-tees" and we consistenly ranked between 1 and 3 for cotton tees during that time. We modified the homepage URL because it seemed spammy and are now just "brandname.com". Does it make sense to go back to the URL with the key phrase in it if that is our main product and we want to rank for it?
Intermediate & Advanced SEO | | EileenCleary0 -
Is a different location in page title, h1 title, and meta description enough to avoid Duplicate Content concern?
I have a dynamic website which will have location-based internal pages that will have a <title>and <h1> title, and meta description tag that will include the subregion of a city. Each page also will have an 'info' section describing the generic product/service offered which will also include the name of the subregion. The 'specific product/service content will be dynamic but in some cases will be almost identical--ie subregion A may sometimes have the same specific content result as subregion B. Will the difference of just the location put in each of the above tags be enough for me to avoid a Duplicate Content concern?</p></title>
Intermediate & Advanced SEO | | couponguy0 -
Can videos be considered duplicate content?
I have a page that ranks 5 and to get a rich snippet I'm thinking of adding a relevant video to the page. Thing is, the video is already on another page which ranks for this keyword... but only at position 20. As it happens the page the video is on is the more important page for other keywords, so I won't remove it. Will having the same video on two pages be considered a duplicate?
Intermediate & Advanced SEO | | Brocberry0 -
Duplicate Content on Product Pages
I'm getting a lot of duplicate content errors on my ecommerce site www.outdoormegastore.co.uk mainly centered around product pages. The products are completely different in terms of the title, meta data, product descriptions and images (with alt tags)but SEOmoz is still identifying them as duplicates and we've noticed a significant drop in google ranking lately. Admittedly the product descriptions are a little bit thin but I don't understand why the pages would be viewed as duplicates and therefore can be ranked lower? The content is definitely unique too. As an example these three pages have been identified as being duplicates of each other. http://www.outdoormegastore.co.uk/regatta-landtrek-25l-rucksack.html http://www.outdoormegastore.co.uk/canyon-bryce-adult-cycling-helmet-9045.html http://www.outdoormegastore.co.uk/outwell-minnesota-6-carpet-for-green-07-08-tent.html
Intermediate & Advanced SEO | | gavinhoman0 -
Duplicate content: is it possible to write a page, delete it and use it for a different site?
Hi, I've a simple question. Some time ago I built a site and added pages to it. I have found out that the site was penalized by Google and I have neglected it. The problem is that I had written well-optimized pages on that site, which I would like to use on another website. Thus, my question is: if I delete a page I had written on site 1, can use it on page 2 without being penalized by Google due to duplicate content? Please note: site one would still be online. I will simply delete some pages and use them on site 2. Thank you.
Intermediate & Advanced SEO | | salvyy0 -
Should I do something about this duplicate content? If so, what?
On our real estate site we have our office listings displayed. The listings are generated from a scraping script that I wrote. As such, all of our listings have the exact same description snippet as every other agent in our office. The rest of the page consists of site-wide sidebars and a contact form. The title of the page is the address of the house and so is the H1 tag. Manually changing the descriptions is not an option. Do you think it would help to have some randomly generated stuff on the page such as "similar listings"? Any other ideas? Thanks!
Intermediate & Advanced SEO | | MarieHaynes0