Diagnosing duplicate content issues
-
We recently made some updates to our site, one of which involved launching a bunch of new pages. Shortly afterwards we saw a significant drop in organic traffic. Some of the new pages list similar content as previously existed on our site, but in different orders. So our question is, what's the best way to diagnose whether this was the cause of our ranking drop? My current thought is to block the new directories via robots.txt for a couple days and see if traffic improves. Is this a good approach? Any other suggestions?
-
I hate to advocate full-scale blocking, but if you really took a hit, and you know the timeline coincided with the new content, it is possible. It might be better to scale back and re-roll out new content in chunks.
One warning - if this is a regular filter (you added a bunch of duplicates), Google should start re-ranking content as soon as the blocking kicks in (this may take weeks, not days). If this was Panda-related or more severe, though, it could take a month or more to see an impact. Not to be the bearer of bad news, but don't Robots.txt block the pages for 2 days, decide it didn't work, and unblock them.
A slightly less extreme approach would be to META NOINDEX all of the pages. That way, you could start to selectively lift the NOINDEX on content piece by piece. If you Robots.txt block all the new directories, it's going to be hard to re-introduce the content. You'll end up releasing the block all at once and potentially just having the same problem again.
-
Could always give it a whirl...
-
yeah, unfortunately we've checked all the items you listed and there's nothing obvious. seems like blocking pages is the only option left..
-
Here's what I'd do.
First, check big stuff like:
- Did your URLs change? Is so, were redirects applied?
- Did your devs accidentally noindex or nofollow entire sections of the site? I've seen this happen a bunch.
If neither of the above, I'd next dig deep with analytics to try to figure out if there is a certain pages, type of pages (product page, articles, blog, etc), etc. that have taken the biggest hit.It may give you more direction. Could also:
- Review Webmaster tools
- Run a site scanner link ScreamingFrog and look for big errors
You could always try blocking new stuff but you never know how long it's gonna take google to come back and honor your changes.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Multiple H2 with no direct links to content and invisible body text - is this an issue?
OK, so we've lost pagerank and I think it's because of the way our site works (and we operate it). We have a videofolio, which shows on most of our pages, showcasing our work. Over time, we have tended to unpublish these pages and created new videofolio pages to show on our home page and relevant pages. The videofolio is a set of pages, each with a title, body text and a place to insert a link to the video, which is played through a videofolio showcase on our website (www.curveball-media.co.uk). Each is set a category, e.g. film, and when the user clicks the tab for 'film', the thumbnails pop up and the user can play the video. We have to work it this way as it's the only way to remove the videos from showing on our home page and to show new content instead. Simply deselecting a category still allows the videos to be seen when the 'all' category is selected by the user. Last week, I found a way of bringing back these unpublished pages by removing the 'all' tab from the videofolio. Then I turned each one into a blog like structure instead. Essentially, instead of the video link being played through the videofolio, we deselected a category (e.g. animation, film etc) and left the page floating. The only way you can access it without being attached to a videofolio category is through the direct link. By turning off the 'all' category and deselecting the page from any other categories, we were able to properly SEO these pages. NB: If they are created for use with the videofolio, you can have only extremely limited body text and no H2, as this is the text that appears when you hover over the video thumbnail. That's just the nature of the template. What I didn't anticipate is that now the code on the home page shows all these now (un)published pages and their corresponding H2 tags. Without a category selected, there is no way to get to these pages unless I create a direct link. I plan to do this through a blog post. In the home page code, the entire videofolio page shows, including the body text and link to the video. **This text doesn't show on the home page though, i.e. the user never sees this text. ** 1. Is it an issue to have so many similar H2 tags on the homepage? 2. Is it an issue that the code has text which is essentially invisible on the home page? 3. Is it an issue that the content is not linked to through the home page visibly? Thanks!
Intermediate & Advanced SEO | | curveballmedia0 -
Including FAQ as Invividual Blog Posts Without Duplicate Issues
My website's FAQ section has a lot of detailed answers, of which I want to upload most on an individual basis to my blog. Example: I may have 30 FAQ and I want to upload 28 of these FAQ as individual blog posts, as it could be good additional search traffic. Question: how do I deal with duplicate content issues? Do I Include canonical? The FAQ are all on the same URL - not separate URL's - which means each blog post would only represent a small % of the entire FAQ section, though each blog would be a 100% copy of an FAQ.
Intermediate & Advanced SEO | | khi51 -
Duplicate content
I run about 10 sites and most of them seemed to fall foul of the penguin update and even though I have never sought inorganic links I have been frantically searching for a link based answer since April. However since asking a question here I have been pointed in another direction by one of your contributors. It seems At least 6 of my sites have duplicate content issues. If you search Google for "We have selected nearly 200 pictures of short haircuts and hair styles in 16 galleries" which is the first bit of text from the site short-hairstyles.com about 30000 results appear. I don't know where they're from nor why anyone would want to do this. I presume its automated since there is so much of it. I have decided to redo the content. So I guess (hope) at some point in the future the duplicate nature will be flushed from Google's index? But how do I prevent it happening again? It's impractical to redo the content every month or so. For example if you search for "This facility is written in Flash® to use it you need to have Flash® installed." from another of my sites that I coincidently uploaded a new page to a couple of days ago, only the duplicate content shows up not my original site. So whoever is doing this is finding new stuff on my site and getting it indexed on google before even google sees it on my site! Thanks, Ian
Intermediate & Advanced SEO | | jwdl0 -
Duplicate content even with 301 redirects
I know this isn't a developer forum but I figure someone will know the answer to this. My site is http://www.stadriemblems.com and I have a 301 redirect in my .htaccess file to redirect all non-www to www and it works great. But SEOmoz seems to think this doesn't apply to my blog, which is located at http://www.stadriemblems.com/blog It doesn't seem to make sense that I'd need to place code in every .htaccess file of every sub-folder. If I do, what code can I use? The weirdest part about this is that the redirecting works just fine; it's just SEOmoz's crawler that doesn't seem to be with the program here. Does this happen to you?
Intermediate & Advanced SEO | | UnderRugSwept0 -
Duplicate WordPress Home Page Issue
I have an issue where I've created a site, (www.tntperformance805.com), using WordPress as a CMS. I enabled the option to use a static page as the home page, and created that page as /home. Well, now the issue that exists is that Google is indexing both www.tntperformance805.com, and www.tntperformance805.com/home/. I've already setup a 301 redirect, pointing /home/ to the main domain, and even have rel=canonical set up automatically, pointing every page to the www version of that particular page. However, Google Webmaster Tools is still reporting the pages as having duplicate page titles and descriptions. I've even had the page removed from Google's cache and index. I'm assuming Google is not considering the 301 redirect, even though it's setup properly. Should I add rel="canonical" href="http://www.tntperformance805.com" /> to the body of the /home/ post, to ensure that it is giving credit to the main domain? I am assuming the page is only redirecting to , as that's the www version, but I thought the 301 redirect would enforce that the search engines should give all credit to the main domain. Thanks in advance for the help everyone. I look forward to some insightful feedback. Best Regards, Matt Dimock
Intermediate & Advanced SEO | | National-Positions0 -
Avoiding duplicate content on an ecommerce site
Hi all, I have an ecommerce site which has a standard block of text on 98% of the product pages. The site also has a blog. Because these cause duplicate content and duplicate title issues respectively, how can I ever get around this? Would having the standard text on the product pages displayed as an image help? And how can I stop the blog being listed as duplicate titles without a nofollow? We already have the canonical attribute applied to some areas where this is appropriate e.g. blog and product categories. Thanks for your help 🙂
Intermediate & Advanced SEO | | CMoore850 -
ECommerce products duplicate content issues - is rel="canonical" the answer?
Howdy, I work on a fairly large eCommerce site, shop.confetti.co.uk. Our CMS doesn't allow us to have 1 product with multiple colour and size options so we created individual product pages for each product variation. This of course means that we have duplicate content issues. The layout of the shop works like this; there is a product group page (here is our disposable camera group) and individual product pages are below. We also use a Google shopping feed. I'm sure we're being penalised as so many of the products on our site are duplicated so, my question is this - is rel="canonical" the best way to stop being penalised and how can I implement it? If not, are there any better suggestions? Also, we have targeted some long-tail keywords in some of the product descriptions so will using rel-canonical effect this or the Google shopping feed? I'd love to hear experiences from people who have been through similar things and what the outcome was in terms of ranking/ROI. Thanks in advance.
Intermediate & Advanced SEO | | Confetti_Wedding0 -
Pop Up Pages Being Indexed, Seen As Duplicate Content
I offer users the opportunity to email and embed images from my website. (See this page http://www.andertoons.com/cartoon/6246/ and look under the large image for "Email to a Friend" and "Get Embed HTML" links.) But I'm seeing the ensuing pop-up pages (Ex: http://www.andertoons.com/embed/5231/?KeepThis=true&TB_iframe=true&height=370&width=700&modal=true and http://www.andertoons.com/email/6246/?KeepThis=true&TB_iframe=true&height=432&width=700&modal=true) showing up in Google. Even worse, I think they're seen as duplicate content. How should I deal with this?
Intermediate & Advanced SEO | | andertoons0