Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
You should submit the domain1 to domain2, let the redirect handle the /index.php
We have a domain that is listed as domain1.com to domain2.com but on domain2 you automatically get redirected to the /catalog directory. This has had no known ill effects. In sense you are in fact domain2 you just don't use it as your root.
I hope that makes sense,
Don
Hi smoki666
I can't say for sure but here are a couple things I noticed:
First you are competing with yourself for keywords
Your homepage and your tigi specific page use the same main keywords. I didn't dig further but this should be avoided.
Second your Domain Authority (DA) / Page Authority (PA) is low compared with those ranking above you. To catch up may require time and effort link building.
I hope my points are not taken as criticism rather helpful advice
I think it depends on the source. We typically get a code from Google every 6 months or so, and every now and then I come across different codes from various other services. No problems using them, however if I try and stack codes from the same source / type to the same account they don't work.
HI Alberto,
Understanding the purpose of the NOFOLLOW tag is what would make your decision. In general you should NOFOLLOW.
Edit for Reference: Google NoFollow Tag
In your case you have "resources" which in theory adds value to the page and the users experience. A single page with resource links is not going to hurt your site at all. Without seeing the exact page I would lean to say it would be fine as follow links. That being said, you want it to be tasteful non-spammy and indeed a boon to the user experience and not just a big list of garbage.
Hope this helps,
Don
Okay so you need to use the root to verify the domain first.
Just stop the redirect to /index.php long enough for you to insert Google's verification code. Then turn back on the redirect. What google is trying to accomplish is to verify you actually do own the new domain. Since some domains share directories this is the only way for them to know you do own the domain since you can verify you control it at root.
Hope that helps,
Don
Wish you the best of luck. It is a very good thing you know about your field, that will lend authority to your site if you can translate your knowledge into helpful information on your website.Even the best SEO's if they don't know the industry will struggle, that gives you a leg up.
Good luck
Yes if it is a follow link. The anchor text helps the search engine understand what the page they are linking to is, but the link itself will pass link juice. Therefore your page authority and domain authority will increase which are known factors in SERPs.
Hope that helps,
Don
Hi Sunit,
Stephanie Chang wrote a nice blog post about how to handle this situation here on Moz.
She points out there is no right way to do this. I however really like the option to leave the page and content on the site but mark the product page as out of stock, or unavailable right now. I used to run my own e-commerce store and I had many items that were purchased from overseas. At times I could not get an accurate re-stock time from my suppliers due to language and international challenges. My solution was to create and out of stock (email me when available again) notice. I found when those products did come back in stock, that started selling again quickly do to the emails that went out.
The other thing Search Engines don't particularly like content that comes and goes so that may be why you noticed a hit in some pages.
Hope that helps,
Don
Humm, certainly should contact customer support.
You can get a ton of good information by running a crawl test. I usually use my credits to test competition but in a pinch I could see the benifit of spending a credit on my own sites.
http://pro.seomoz.org/tools/crawl-test
Hope that helps!
Hi Steve,
It is pretty well known duplicate content is not a "good thing" but it doesn't mean it will hurt you directly. Google rarely will penalize a site for duplicate content REF: Matt Cutts
It does however make ranking a specific page harder when you're competing with yourself. Most SEO tools will warn you about each case of duplicate content it finds which can also be annoying.
In the best case scenario you may rank multiple pages for longer tailed keywords for example.. something like
Main Keyword: Gourmet Shopping
Tails: Dayton Ohio, Tampa Florida, Los Angles California, Topeka Kansas
In such case you may would rank 1 of those pages over the other for Gourmet Shopping however if there was enough specific location specific content you may find other pages ranking well in those city state type searches.
Hope that makes sense,
Don
Then 404 is the correct way to handle it.
You should also be sure that you clean up any internal links to these pages after they are gone. Search engines can understand other sites linking to pages that are no longer available but they do frown on your own site sending users to missing pages.
Each campaign can have the same keywords.But will use a keyword slot no matter what keywords you use.
The rational... If you're tracking multiple sites and the same keyword for each site, you are actually tracking the keyword 3 times, just for different domains.
You may want to rely on more then just one site's DA metric to make your actual assessment. Moz's number while helpful is just a very educated guess; which can not take into account all the factors. What is most important is conversion rates, sales, or keyword rankings depending on what your goal is.
It is not uncommon for our DA to go up or down a few points month to month. This usually has a correlation to how many linking domains and the like the crawl comes back with. Not ever site will get scanned every crawl, so this number doesn't play a huge role in how successful I view my efforts.
Hope that helps,
Don
Hi George,
To the first issue. Have you considered using a REL=CANONICAL tag on this pages with thin to no content (point to them to the parent category). Barring that option you could also just put a NOINDEX / NOFOLLOW. No sense in removing the page just in case it has a few links out there.
For the 404 page, the main reason I would suspect Moz seeing this as duplicate content would be because of parameters on it. It would make more sense to have a universal 404 Page and let the server handle the error for you and redirect where appropriate. Moz site.. www.thisSite.com/catalog/ImGoingNoWhere.php
404 Page are usually pretty simple to setup content wise. Your second goal would be to remove all links on your site that would return a 404.
Is it possible to just use Excel's data tab "Text To Columns" and split the data on commas or whatever the delimiter is?
I haven't had this issue before but that may help with the CSV if you use Excel.
If you mean a link like:
VS
Then yes, because the first one is not technically a link.
Hi Dan,
If these pages have no SEO value then you can just stop them from being crawled, thus preventing any duplicate content penalties. If you see some backlinks (SEO value) to any of these then I would use Canonical.
robots.txt
User-agent:: *
Disallow: /*attachment_id
Hope it helps,
Don
I believe that is because it is showing internal linking as well.
Edit:
What I mean here is even though the domain is a subdomain, if you link from the main domain to the subdomain it appears as another domain is linking to you.
Actually no I wouldn't recommend that.
The reason is if the link is helpful it should be a link right? From a users point of view do you not find it frustrating to see a link that is not a link?
My suggestion is to evaluate each site you're linking to, if they deserve the link leave it in. Otherwise simply remove the link.
The reason behind my suggestion is the way the internet and page rank / authority is supposed to work. When a web master find a link to a site that is beneficial to their sites users, then they link to them. This generates page authority to the linked site but also helps the web master serve their users. In turn it also associates the web masters site with the linked site.
The web has taken many twist and turns since the original method of passing link juice was developed. Tools such as robots.txt, nofollow noindex, and disavow have been added to deal with the changing environment. But, the core of the system still remains.
Hope this make sense,
Don
Gotcha, definitely don't want to nix pages then. I would imagine Lynn's response is more appropriate then, it is likely that he is using a plugin that has been updated to better SEO practices that he hasn't yet updated.
Great answer as usual Takeshi.
The only thing I might add, is you should also evaluate the industry leaders. If they are using grey / black hat tricks then you maybe in an industry that needs a real seo leader. Recently I have seen this problem with certain industries like payday loans. There maybe an opportunity here for somebody with strong seo skills and white hat knowledge backed by solid content and great developers to produce a site that crushes it. Thus causing the industry to follow suit and clean up its act, when that happens those using black / grey hat ticks will be decimated.
HI Alberto,
Keep in mind I have not seen your page. I am talking in general specifics. The over all point is if you have a link to another site which is there for reference the user there is no reason to not make it an actual link. Any reason you can think of not to make it a link would also be a reason to remove it.
If you feel you have too many links on a specific page then you probably do! Pick the most pertinent ones and axe the others.
There are ways to maximize link juice and page authority by using some more advanced SEO tactics, See Rand's post about link sculpting. I will say this is some advanced level planing and not something you would just single out one page to do.
Remember SEO stands for Search Engine Optimization. White hat SEO deals with how you can best present your page to search engines with out frustrating your users. When you purposely make a change that negatively effects your users and possibly tricks search engines to rank you better you have crossed into grey or black hat SEO. Something that will eventually bit you in the ass.
The choice is of course yours, and if you would like me to look at the page in question you can PM me a link I will be happy to do so. I do stand by everything I said in all my replies while speaking in general terms.
Don
The alternative he suggested was to delete the page. 
Before I removed a page I would canonical it.You can always undo the Canonical once content becomes available.
Hi Zoe,
I just tried to view your Robots.txt file and was unable to access it. So I requested a crawl. The error I got from Moz is "612 : Page banned by error response for robots.txt."
I would suggest you look at the file permissions of your robots.txt file. It should be readable by the public; Apache permissions would be (644). This means it is not available to view or it could be that it is in the wrong directory or non-existent altogether. It should be in your root and accessible via a browser here: http://www.guitarcontrol.com/robots.txt
I would also note that some people feel that listing your sensitive directories in the robots.txt file is a security risk, and perhaps they are right. However, there are other ways to secure a directory and keep them out of the Indexes, and if the information being protected is that important, I would imagine that all security precautions would be taken.
The robots.txt file is important for crawlers so it should always be used and available to be read. If security is the problem here, I would point to some high profile sites that don't seem to have a problem with showing the public their robots.txt file.
https://www.google.com/robots.txt
https://www.facebook.com/robots.txt
I hope this helps,
Don
<colgroup><col width="907"></colgroup>
| 612 : Page banned by error response for robots.txt. |
That is a nice post Danny.
For a non-developer the scores seem inline and accurate. If you have an existing site and just want to integrate a shopping tool or platform the list is very much how I would rate them.
If you're a developer and want to self customize Zen or Oscommerce would be the best choices IMO. Many out of the box site software are pretty restrictive and tend to lend their selves to a cookie-cutter look and feel, I'm all for customization.
It takes all kinds :). I think to me maybe a learned response any time I click a link and see a redirect unless I implicitly trust the site I'm alt+f4'ing right away. I don't want popups, I don't want redirects I want to go where I say to go, even if where I said to go is empty. At least I got there and now can make a conscience choice what to do next.
Maybe just preference.
Hi Edmond,
It sounds like what you're looking for is called "Who's Online". Not sure about the "pop-up" feature you asked for, but Who's Online should at least give you the data you want.
WP link: https://wordpress.org/plugins/tags/whos-online
Good Luck,
Don
HI Edward,
I'm not sure the "best way" but I can share 3 ways I know of.
Competitor Research
Check your competitors see if they actively participate or simply get links from domains which are forums. Usually a doing a search on their backlink profiles for "blog", "forum" will identify these quickly.
Search Engine Search
I sometimes find relative forums simply buying going to Google typing a broad keyword + forums. At this point I have to evaluate the forum to see if it worth trying to engage with.
Industry Engagement
If you are actively engaged daily in the industry talk you will pick up on what people are posting. For example following industry leader's Twitter, Facebook, Reddit, Instagram profiles will lead you directly to some post and the places they are referencing.
Once you have at least one location worth engaging with, you can also watch those places daily as many times people will re-post information from a similar site / forum. Take for example here.. we oft see info from SearchEngineLand.
Hope this helps,
Don
How are you merchant prices / info being displayed on your site? From your site or using IFrames?
Also Karl, Alexa does a fairly good job at tracking inlinks with referring URL's which is pretty helpful.
Your site is here: http://www.alexa.com/siteinfo/wholesaleclearance.co.uk
And of course Open Site Explorer here will give you some details as well.
Hello Rich,
I think giving them a copy of you domain would be a very bad choice. They will likely have the duplicate content problems as I am sure Google would "most likely" recognize you as the source of the content. Since you were around first. Basically, to Google, the site would be relegated to a low quality clone of yours and they would never see respectable SERP results. Ultimately not really helping either of you.
I am of the opinion there is an alternative solution out there, some off the top of my head....
The actual solution I would recommend would require more information from both sides of the situation, The good thing here is you are in the power seat and nothing really can get done with out your say-so. I would recommend caution with giving anybody direct access to your content, if somebody wants it that means you are doing something right.
I hope this input helps you. I have had personal experience with duplicating a retail website for business to business sales. If you would like more thoughts feel free to PM me.
Don
It looks like something is installed and configured improperly.
You have 2 head tags on the page that shows up from the redirect.
This is actually inside the first head tag complete with a body tag and another doc declaration.
<iframe id="oauth2relay579972146" name="oauth2relay579972146" src="https://accounts.google.com/o/oauth2/postmessageRelay?parent=http%3A%2F%2Fwww.incipio.com#rpctoken=728288212&forcesecure=1" style="width: 1px; height: 1px; position: absolute; top: -100px;" tabindex="-1">
<html><head><title>title><meta content="text/html; charset=utf-8" http-equiv="content-type"><meta content="IE=edge" http-equiv="X-UA-Compatible"><meta content="width=device-width, initial-scale=1, minimum-scale=1, maximum-scale=1, user-scalable=0" name="viewport"><script src="https://apis.google.com/js/api.js" type="text/javascript" gapi_processed="true"><script src="https://oauth.googleusercontent.com/gadgets/js/core:rpc:shindig.random:shindig.sha1.js?c=2" type="text/javascript"><script src="https://ssl.gstatic.com/accounts/o/3417060037-postmessagerelay.js">head><body>html>iframe>
That looks like an installation issue.
Now the misconfiguration issue would have to be why the URL re-writes to page but serves up different content.
And lastly I think even if you fix those issues you're still going to get duplicate content warnings because you have very thin content on pages.
Example: Page 1 http://www.incipio.com/cases/tablet-cases/amazon-kindle-cases-sleeves/amazon-kindle-fire-hd-6-cases.html
Example: Page 2 http://www.incipio.com/cases/tablet-cases/amazon-kindle-cases-sleeves/amazon-kindle-fire-hd-7-cases.html
On those 2 pages there is a 1 character difference 6 instead of 7. All the other content (header & footer) and 1 letter difference. Than if you go to the actual product page you have the exact same issue same description to the letter except the one number. Yep, you're going to have a duplicate content problem.
This is something that all e-commerce stores face. You honestly need to write unique content for each and every product you sell. Don't copy & paste stuff from another site like Amazon or the manufacturers site, write your own content.
In summation, I would recheck any modules/ad-ons/plug-ins you installed as one appears to be incorrect. if that doesn't' fix the re-write issue have a developer that is familiar with your ecommerce platform look at this issue. Lastly, you got to have unique content.
Maybe not the best news but I hope it helps
Don
Edit in bullet points to try and make the post a look a little better. These forums don't take kindly to adding code blocks 
Hi Chris I've witnessed the same thing in a couple other markets. Likely, their results are due to an aggressive link strategy that will ultimately bite them in the A**. The two sites I noticed this from held their spots for 1 week and the other 2 1/2 weeks, after which they disappeared from the first 2 pages.
One of them is one we track here at SEOMoz.org I included is an image of when we noticed this site rank very high, over-layed on their links over time graph.
Hi Rod,
I will try address the 2 questions I see here.
1. Is an "index stack" bad for your site.
2. How do I optimize my site.
1. Index Stack, okay to be completely honest I have never heard the term before. However, with the link you provided I was able to understand what you mean. In such case you have taken the content of your sub-pages and placed them directly on your home page. Is this bad.. YES!
The reason why this is bad is because it is creating duplicate content. This is bad because the page that should be ranking for X content is now competing with your home page for these rankings. Your home page should be about your business and services, what you do, and how you do it. The sub-pages are your re-enforcement they support the content stated on your homepage.
Imagine a beautiful hotel. The outside is adorned in the finest materials of the time, well maintained and awe inspiring. A equally impressive doorway beacons you to come in. As you do, you find yourself in the lobby. Well placed pristine signs detail some of the hotel's offerings, private gym's, hot tubs, saunas, massage, 5 star dinning, dance halls, and judging by the impressive lobby you have no doubt these offerings will be spectacular.
In this example the lobby is your homepage an impressive list of the offerings guest will find inside. What you don't see is a middle-aged fat man sprawled out in the lobby getting a massage. Which is what your index stack is doing.
2. How to optimize (fix it). To fix this you'll want to structure you home page to be about your business, and services. High level overviews of what you offer, with strong brand placement.
The purpose of the homepage is to make it clear what you do, how you do it, why people love it, and in general more about your brand. The purpose of your inner pages is to completely detail the particular service or products, with call to action to convert interest into money.
I would remove the internal page content from the homepage, and focus again more on an overview. Essentially cut out everything between the Hunting & Gathering Club, and the Call to Action "There's a Southern Whitewater Experience for everyone". Beef up the links to the internal pages by making them look more attractive and obvious.
There are other things that may need addressed as well, but I hope this answers your primary question,
Don
WordPress is good at creating duplicate content. I would check at the official site to see if there is way to disable the archiving "tag" feature as it appears to be redundant.
Andreas recommendation would work as well, simply check if you're at an archive tag page, if so then display the meta tag noindex.
You also could do this inside of a robots.txt
User-agent: *
Disallow: /tag/*
This would effectively stop any pages from being crawled when the crawler sees the "tag" directory.
Hope that helps,
Don
HI Chris
There is a way to track it, checkout the campaigns in Campaign section. You can add 3 of your competitors there and set up the keywords you want to watch. Unfortunately, the there is no great way to go back in time, but by tracking them now you can get a feel for what happens over upcoming weeks.
Thanks for sharing.
This is an interesting case for me. On one hand by writing technical no keyword content he likely drew a lot of technical links to the site due to depth. Links from high authoritative sites like Colleges and Government sites.
On the other hand it may have limited appeal to the general population due to the technical nitty gritty, where they just want to know if "X" firm handle said type of cases and what kind of cost is associated with it. But I could also see how more important high profile clients may find the technical stuff more important as cost is less of an issue.
All and all this was a good read.
Thank you,
Don
There would not be a direct SEO benefit for doing this. There maybe however a benefit in tracking. If you only used that sub-domain for X ad campaign than you would know all traffic from referral sub-domain would be coming from that ad campaign.
There may be some slight non-optimization for doing it this way. Sub-domains are treated as their own domains to a degree, so you are in affect giving the ad-campaign's link to juice to a new domain entirely. Then forwarding that to a specific page. Opposed to just directly giving the link juice an ad campaign can generate to the actual page.
A couple things here depending on the type of ad campaign there may not be any link juice to worry about, like Google's ad words don't pass link juice. However, if you purchased direct advertisement on certain sites you may get some link juice from those ads running.
The second thing is actually a question. What is the purpose of creating a sub-domain to point to a sub directory? Is it just for tracking? Or were you wondering if you could benefit from a sub-domain being treated as a new domain linking to you? If for tracking; I would think there are other tracking methods that could handle referring traffic. If it were in hopes of gaining a new backlink from a different domain than I would say it isn't helpful this way. First because it is simply forwarding to the sub-directory and secondly even it weren't forwarding the link would be considered from the same server and not very helpful anyway.
So in short, no benefit other than a potential way to help with tracking.
Hope that makes sense and helps,
Don
edit some grammar
Hi Snowflake,
You can use the OSE (Open Site Explorer) here on Moz to check the links they found. You can download that report to CSV to easily sort and see if you have a possible negative campaign running against you.
You could also use, Majestic, or SEMRush. to find more links. Just note there is no tool, free or paid that is going to be able to get all the links pointing to your site.
If you don't find a lot of spam links to your site, chances are there isn't somebody trying to target you with a negative campaign.
Hope it helps,
HI Rod,
That should have fixed your duplicate content problem.
Hi David,
Rand covered this very topic in a white board friday. Perhaps you may find it helpful and provide insight on what can happen and why he thinks the way he does.
Hope it helps,
Don
Hi,
There are several reasons.
If you have recently changed your url structure. IE (went from using www to not, or https or not, or trailing / or not). In these cases Google could have indexed the pages already under the "other" version.
Google could be having a crawling error, like in a robots.txt or lack there of. Improper canonical tags, blocked access, improper redirects, or a manual penalty.
If you would like to post a link (or pm me) I will take a look and see if I can spot a potential problem for you.
Here are a couple links on Google that should help:
Why Pages Drop From Index
Overview Pages Not Being Crawled
Hope this helps,
Don
HI Veebs,
So the first thing I did was go to the proper link.
[link removed for privacy reasons]
Then I tried going to the /self/self link.. and yep its a valid page.
This tells me that something server side is generating these links. Was hard to do much more then look at the code as you have a pop-up register splash that prevents me from looking at it element by element. Which maybe part of the issue. However you should be able to just put a canonical on each of your pages to the correct url to prevent Moz from seeing all the /self/self/self as duplicate, but more importantly from those pages being indexed incorrectly.
Hope it helps,
Don
Hello,
This is a very common challenge in Ecommerce.
You have a couple options to consider.
1. Keep the existing format but add more unique content to each based on the type. For example on the Powerpoint template you could add at least a paragraph on PowerPoint and how this template works inside the program, a video of you downloading it, opening it, and modifying it with some test data.
2. As you pointed out, one page with 3 different download options.. you can even use this option to up-sell a bundle which includes all 3.
As it is now Google is going to pick one of those pages and index it and basically ignore the others as duplicate content. You may actually have further issues throughout the site due to the shear volume of similar content on each page (the bulk of the page's content is exactly the same baring the item's description). You may need to evaluate how you present all the other info on this page (how it works, what others are saying, whats in this template, whats included etc..) all this is leading to many pages with a lot of the exact same content. If other of your pages are displaying all this info as well, the one paragraph description may not be enough to make the page unique to a search engine.
You can review your site on Siteliner here.
Again this is a big challenge for most Ecommerce sites, especially those with a lot of products. However, for smaller sites with less products it is important to focus on each one of these products and write unique, compelling, informative content that sets you apart from the larger sites.
Hope this helps,
Don
So I'm not seeing anything blocking crawling on your site which is good. But I did notice that you have at one time used URL types "http" and "https" which leads me to believe you may have recently switched to HTTPS. In such case you should know that it may take Google sometime to adjust. On a technical level, https and http are 2 different domains.
It is highly likely that Google has index the HTTP version of some of these pages which is why your index count maybe lower then normal for the HTTPS version.
I do see you properly 301 redirected these pages and your sitemaps are reflecting the https as well, if again this was a recent change it just looks like its going to take a bit of time for Google to catch up.
This is worth a quick look, https://support.google.com/webmasters/answer/6073543?hl=en (scroll to the bottom) and see the section "Migrating from HTTP to HTTPS".
I sent you additional info in PM.
Hope this helps,
Don
Hi Ruchi,
Can you provide an example of a page and the expected page?
There could be a number of reasons why this is happening, for anybody to get specific we need to see what we are working with.
Thank you,
Don
HI Liam,
Yes you can fix this with a canonical tag or you can do a blanket fix that redirects all "/" urls to the non-slash version with .htaccess.
This should work in the .htaccess file
RewriteRule ^(.*)/$ /$1 [L,R=301]
Hope it helps,
Don
Also a quick point, if you still have your Google search console setup for HTTP, even though you use HTTPS now, I'd suggest looking at what is being reported as indexed in there. That maybe the missing link.
Cordialement,
Don
Hello Bob,
Here is some food for thought. If you disallow a page in Robots.txt, google for example will not crawl that page. That does not however mean they will remove it from the index if it had previously been crawled. It simply treats it as inaccessible and moves on. It will take some time, months before Google finally says, we have no fresh crawls of page x, its time to remove it from the index.
On the other hand if you specifically allow Google to crawl those pages and show a no-index tag on it, Google now has a new directive it can act upon immediately.
So my evaluation of the situation would be to do 1 of 2 things.
1. Remove the disallow from robots and allow Google to crawl the pages again. However, this time use no-index, no-follow tags.
2. Remove the disallow from robots and allow Google to crawl the pages again, but use canonical tags to the main "filter" page to prevent further indexing the specific filter pages.
Which option is best depends on the amount of urls being indexed, a few thousand canonical would be my choice. A few hundred thousand, then no index would make more sense.
Whichever option, you will have to insure Google re-crawls, and then allow them time to re-index appropriately. Not a quick fix, but a fix none the less.
My thoughts and I hope it makes sense,
Don
No problem Liam,
You can safely canonical your home page to itself. I've had 2 sites setup that way for a few years now.
Good luck,
Don
