Duplicate Content Resolution Suggestion?

[PlasticCards]

SEOmoz tools is saying there is duplicate content for:

www.mydomain.com

www.mydomain.com/index.html

What would be the best way to resolve this "error"?

[webtarget]

Does having the line:

DirectoryIndex index.html

Have any use in addition to the lines you posted?

Thanks.

[sferrino]

Stephen I agree with the KISS method. Using an htaccess RewriteCond is not the simplest solution for someone who does not know htaccess syntax. In an effort to fully answer this, here is the typical code we are referring to:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^yoursite.com
RewriteRule (.*) http://www.yoursite.com/$1 [R=301,L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index.html\ HTTP/
RewriteRule ^index.html$ http://www.yoursite.com/ [R=301,L]

The first 2 lines are typical commands to Follow Symbolic links, and make sure that the rewrite engine is in the on state.

The first RewriteCond looks at the host, and if it is not the www. version the RewriteRule will redirect the visitor to the http://www. version of your site.

The second RewriteCond looks at whether this is an index.html file, if it is, it will the RewriteRule will 301 redirect them to the version with out the index.html, just yoursite.com/

[firstconversion]

Im a KISS guy, duplicate content pages should just be handled with a rewrite, then they don't  appear in any of your stats, attract links, spread your like/tweet numbers over multiple pages, if you are using xml files to keep tabs on your indexing etc and give you a better idea of whats going on on your site

Also, you have to take into account Facebook likes, page tweets, +1s etc - does rel canonical work on the social graph data?

rel canonical sort orders etc but if its a pure duplicate, then 301

"Dont link to page X on your site" isnt really a good solution in my eyes, too much room for error

[RikkiD22]

Completely agree.

I think II may have been slightly confused by thinking the default for www.mydomain.com/ was not iindex.html

[sferrino]

Yes, by the original posting your impression is correct and are the same page, but you can't 301 an index.html page to the domain where the index.html is the page that shows by default.

You could use an htaccess RewriteCond, but could be a little overkill for this situation, where adding a canonical will solve it.

[RikkiD22]

I was under the impression that www.mydomain.com/ and www.mydomain.com/index.html were both indexing but are the same page

[sferrino]

If the index.html is their home page that shows up when just doing the domain: http://www.mydomain.com/

then what would you 301 to? Are you assuming that it is a site that is using a index.php, index.htm, index.asp, etc.?

PlasticCards stated that there is a duplicate content, therefore the index.html page actually exists and should use a canonical.

[RikkiD22]

I think in that particular situation I would use 301 as there really isn't a separate use for the /index.html page

[firstconversion]

or 301 redirect in your .htaccess and then you don't have to worry about link issues etc

[sferrino]

Use a rel='canonical' and use the non index.html for the href;

also don't link to the index.html from anywhere.