Duplicate content with same URL?
-
SEOmoz is saying that I have duplicate content on:
The only difference I see in the URL is that the "content.asp" is capitalized in the second URL.
Should I be worried about this or is this an issue with the SEOmoz crawl?
Thanks for any help.
Mike
-
I am not using a rewrite rule yet -- I was asking if there is one that would resolve this issue.
-
Are you specifying the URL rewrite rule at the page level, or in your .htaccess? I had a similar issue once on a WordPress Multisite install that was rewriting
example.com/site2 -> site2.com
And:
example.com/site3 -> site3.comThe issue wasn't "real" in that the users' browsers were moving to the preferred URLs specified in the HTTP headers, but our crawl tests were a nightmare of non-existent files much like yours. Rel="canonical" will help in that case to avoid penalties, but won't do any favors for page rank or indexation. I believe our developers created some additional page-level rewrites to deal with the phantom pages created in the crawl, but alas, I'm not sure what the details were.
You might post in a new thread or reach out to Chris Abernethy directly, he's far savvier with PHP than I am.
-
I have a similar problem, and I couldn't see a solution on the site that your link refers to. Maybe you can help?
In both SEOmoz reports and GWT I get duplicate meta descriptions and/or duplicate title tags on pages that do not physically (or logically) exist. I'm not talking about dynamically generated URLs. What I see is for a given page, several other appended pages that have no relationship to the first, like this:
/realpage1.php/anotherrealpage1.html
/realpage1.php/adifferentrealpage2.html
/realpage1.php/anotherrealpage3.php
/realpage1.php/directory/realpage4.htmlPerhaps related to this issue, I discovered that if a trailing slash is entered after any URL typed into the browser (other than the home page), our custom 404 page appears, but with no CSS styling or active javascript.
I have been wondering if a rewrite rule that eliminates trailing slashes would work, but then it would never display a sub-directory's default index page, right?
I've searched all over for some help with this, to no avail. Any help will be much appreciated.
-
Modern search engines won't penalize you for this, but you may lose link juice if your content has multiple URLs and each is receiving links. Best practice is to set up a few simple PHP mod_rewrite rules in your .htaccess for basic URL display issues (enforce trailing backslash, redirect to/away from www, etc.), as well as to declare your preferred URL in the HTML of each page using this handy .
Here's a great tutorial how to force lower-case URLs written by a fellow Mozzer (props, Chris! It's how I learned...), and here's 10 other useful mod_rewrites to add to your repertoire.
-
You sir are a gentleman and a scholar.
Thanks for your help Matt.
-
Use canonicalization to resolve this common duplicate content issue.
You need to place the canonical tag pointing to your preferred URL.
See this SeoMoz guide on how to do it -
http://www.seomoz.org/learn-seo/duplicate-content
See
Rel="canonical"
this actually uses the example of capitalization and one page appearing as three to search engines...
Hope this helps!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Shopify Duplicate Content in products
Hello Moz Community, New to Moz and looking forward to beginning my journey towards SEO education and improving our clients' sites. Our client's website is a Shopify store. https://spiritsofthewestcoast.com/ Our first Moz reports show 686 duplicate content issues. I will show the first 4 as examples. https://spiritsofthewestcoast.com/collections/native-earrings-and-studs-in-silver-and-gold/products/haida-eagle-teardrop-earrings https://spiritsofthewestcoast.com/collections/native-earrings-and-studs-in-silver-and-gold/products/haida-orca-silver-earrings https://spiritsofthewestcoast.com/collections/native-earrings-and-studs-in-silver-and-gold/products/silver-oval-earrings https://spiritsofthewestcoast.com/collections/native-earrings-and-studs-in-silver-and-gold/products/haida-eagle-spirit-silver-earrings As you can see, URL titles are unique. But I know that the content in each of those products have very similar product descriptions but not exactly. But since they have been flagged as a site issue by Moz, I am guessing that the content is 95% duplicate. So can a rel=canonical be the right solution for this type of duplicate content? Or should I be considering adding new content to each of 686 products to drop below the 95% threshold? Or another solution that I may not be aware of. Thanks in advance for your assistance and expertise! Sean
Technical SEO | | TheUpdateCompany1 -
Crawl Diagnostics: Duplicate Content Issues
The Moz crawl diagnostic is showing that I have some duplicate content issues on my site. For the most part, these are variations of the same product that are listed individually (i.e size/color). What would be the best way to deal with this? Choose one variation of the product and add a canonical tag? Thanks
Technical SEO | | inhouseseo0 -
Duplicate Content Issue WWW and Non WWW
One of my sites got hit with duplicate content a while ago because Google seemed to be considering hhtp, https, www, and non ww versions of the site all different sites. We thought we fixed it, but for some reason https://www and just https:// are giving us duplicate content again. I can't seem to figure out why it keeps doing this. The url is https://bandsonabudget.com if any of you want to see if you can figure out why I am still having this issue.
Technical SEO | | Michael4g1 -
Duplicate Content Issues - Where to start???
Dear All I have recently joined a new company Just Go Holidays - www.justgoholidays.com I have used the SEO Moz tools (yesterday) to review the site and see that I have lots of duplicate content/pages and also lots of duplicate titles all of which I am looking to deal with. Lots of the duplicate pages appear to be surrounding, additional parameters that are used on our site to refine and or track various marketing campaigns. I have therefore been into Google Webmaster Tools and defined each of these parameters. I have also built a new XML sitemap and submitted that too. It looks as is we have two versions of the site, one being at www.justgoholidays.com and the other without the www It appears that there are no redirects from the latter to the former, do I need to use 301's here or is it ok to use canonicalisation instead? Any thoughts on an action plan to try to address these issues in the right order and the right way would be very gratefully received as I am feeling a little overwhelmed at the moment. (we also use a CMS system that is not particularly friendly and I think I will have to go directly to the developers to make lots of the required changes which is sure to cost - therefore really don't want to get this wrong) All the best Matt
Technical SEO | | MattByrne0 -
Noticed a lot of duplicate content errors...
how do I fix duplicate content errors on categories and tags? I am trying to get rid of all the duplicate content and I'm really not sure how to. Any suggestions, advice and/or help on this would be greatly appreciated. I did add the canonical url through the SEO Yoast plugin, but I am still seeing errors. I did this on over 200 pages. Thanks for any assistance in advance. Jaime
Technical SEO | | slapshotstudio0 -
Duplicate Page Content
Hi, I just had my site crawled by the seomoz robot and it came back with some errors. Basically it seems the categories and dates are not crawling directly. I'm a SEO newbie here Below is a capture of the video of what I am talking about. Any ideas on how to fix this? Hkpekchp
Technical SEO | | mcardenal0 -
Development Website Duplicate Content Issue
Hi, We launched a client's website around 7th January 2013 (http://rollerbannerscheap.co.uk), we originally constructed the website on a development domain (http://dev.rollerbannerscheap.co.uk) which was active for around 6-8 months (the dev site was unblocked from search engines for the first 3-4 months, but then blocked again) before we migrated dev --> live. In late Jan 2013 changed the robots.txt file to allow search engines to index the website. A week later I accidentally logged into the DEV website and also changed the robots.txt file to allow the search engines to index it. This obviously caused a duplicate content issue as both sites were identical. I realised what I had done a couple of days later and blocked the dev site from the search engines with the robots.txt file. Most of the pages from the dev site had been de-indexed from Google apart from 3, the home page (dev.rollerbannerscheap.co.uk, and two blog pages). The live site has 184 pages indexed in Google. So I thought the last 3 dev pages would disappear after a few weeks. I checked back late February and the 3 dev site pages were still indexed in Google. I decided to 301 redirect the dev site to the live site to tell Google to rank the live site and to ignore the dev site content. I also checked the robots.txt file on the dev site and this was blocking search engines too. But still the dev site is being found in Google wherever the live site should be found. When I do find the dev site in Google it displays this; Roller Banners Cheap » admin <cite>dev.rollerbannerscheap.co.uk/</cite><a id="srsl_0" class="pplsrsla" tabindex="0" data-ved="0CEQQ5hkwAA" data-url="http://dev.rollerbannerscheap.co.uk/" data-title="Roller Banners Cheap » admin" data-sli="srsl_0" data-ci="srslc_0" data-vli="srslcl_0" data-slg="webres"></a>A description for this result is not available because of this site's robots.txt – learn more.This is really affecting our clients SEO plan and we can't seem to remove the dev site or rank the live site in Google.Please can anyone help?
Technical SEO | | SO_UK0 -
What could be the cause of this duplicate content error?
I only have one index.htm and I'm seeing a duplicate content error. What could be causing this? IUJvfZE.png
Technical SEO | | ScottMcPherson1