External Links from own domain
-
Hi all,
I have a very weird question about external links to our site from our own domain.
According to GWMT we have 603,404,378 links from our own domain to our domain (see screen 1) We noticed when we drilled down that this is from disabled sub-domains like m.jump.co.za.
In the past we used to redirect all traffic from sub-domains to our primary www domain. But it seems that for some time in the past that google had access to crawl some of our sub-domains, but in december 2010 we fixed this so that all sub-domain traffic redirects (301) to our primary domain. Example http://m.jump.co.za/search/ipod/ redirected to http://www.jump.co.za/search/ipod/
The weird part is that the number of external links kept on growing and is now sitting on a massive number.
On 8 April 2011 we took a different approach and we created a landing page for m.jump.co.za and all other requests generated 404 errors. We added all the directories to the robots.txt and we also manually removed all the directories from GWMT.
Now 3 weeks later, and the number of external links just keeps on growing: Here is some stats:
11-Apr-11 - 543 747 534
12-Apr-11 - 554 066 716
13-Apr-11 - 554 066 716
14-Apr-11 - 554 066 716
15-Apr-11 - 521 528 014
16-Apr-11 - 515 098 895
17-Apr-11 - 515 098 895
18-Apr-11 - 515 098 895
19-Apr-11 - 520 404 181
20-Apr-11 - 520 404 181
21-Apr-11 - 520 404 181
26-Apr-11 - 520 404 181
27-Apr-11 - 520 404 181
28-Apr-11 - 603 404 378
I am now thinking of cleaning the robots.txt and re-including all the excluded directories from GWMT and to see if google will be able to get rid of all these links.
What do you think is the best solution to get rid of all these invalid pages.
-
We had 301s for about 6 months, and the old URLs did not disappear from google. Thats why we decided to change them to 404s, with the thinking that Google might remove them quicker. But the number of links from sub-domains just keeps on growing.
I am worried that by having these problem urls listed in the robots.txt actually prevents google from following them and seeing that it should be removed and that it returns a 404
-
Instead of trying to manage a massive 301 list, can you just customize your 404 page to redirect?
{script to test page URL}
$location = "http://www.YourSite.com/";
header("HTTP/1.1 301 Moved Permanently");
header("Location: {$location}");
exit;
}
-
Update:
There are 2 things that still puzzles me with this:
If you go to http://www.google.co.za/search?q=site:jump.co.za+-www&hl=en&rlz=1C1GPCK_enZA426ZA426&prmd=ivns&filter=0&biw=1920&bih=979 you notice all sorts of weird sub-domains, and all of these are invalid and have been removed from GWMT.
If you manage the domain m.jump.co.za on GWMT you also notice that it still reports on keywords, queries and all sorts of data, although the site is disabled and all the URLs generate 404 errors
There is only a few of these weird sub-domains that are causing the problems:
0www.
iiiiiwww.
iwww.
m.
wtfwww.
www.www.
wwww.All these domains feels very fimiliar to me and I am almost 100% sure that its domains that used to test when we found the problem on apache, meaning google took the data from the toolbar queries and probably started indexing these sub-domains. But now I can't get rid of them, and Google seems to be out of control with these.
So the main question is probably, should we just give 404s or should we add to Robots.txt as well?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Using one domain for email and another domain for your website, but redirects...
Hello - We are rebranding and our new name is fairly lengthy. We own all main domain versions of our brand name - .com, .new and .org - There is a very high search volume for the new brand name as it is a merger of 2 popular existing brands so want to take advantage of that and use our full name within our website domain name. However, since the name is a little long as mentioned - 25 characters - we also own the 3 character acronym of the new brand so we are debating on using the acronym for our new email addresses. ie [email protected] so it is user friendly. We would obviously redirect the acronym email domain to point to the longer website domain. Are there any negative SEO effects if we do that? Use the longer domain for the website and shorter acronym for our email? Thank you
Technical SEO | | KRBishopBh1 -
Helping finding a link
Hi So Ive done a crawl of the site using screaming frog. There are a few old category and sub category pages which don't exist any more but somehow the crawler is finding them. An example is below: http://www.ebuyer.com/store/Home-Appliances/cat/Health-&-Beauty/subcat/Male-Grooming Just wondering if anybody had any ideas about how I could go and find these urls and remove them off the site. Any ideas would be really appreciated. Thanks Andy
Technical SEO | | Andy-Halliday0 -
Moving from www.domain.com/nameofblog to www.domain.com/blog
Describe your question in detail. The more information you give, the better! It helps give context for a great answer I have had my blog located at www.legacytravel.com/ramblings for a while. I now believe that, from an SEO perspective, it would be preferable to move it to www.legacytravel.com/blog. So, I want to be able to not lose any links (few though they may be) with the move. I believe I would need to do a 301 redirect in the htaccess file of www.legacytravel.com that will tell anyone who comes knocking on the door of www.legacytravel.com/ramblings/blah blah blah that now what they want is at www.legacytravel.com/blog/blah blah blah Is that correct? What would the entry look like in the htaccess? Thank you in advance.
Technical SEO | | cathibanks0 -
Do I need a link to my sitemap?
I have a very large sitemap. I submit it to both Google and Bing, but do I need a link to it? If someone went there it would probably lock their browser. Is there any danger of not having a link if I submit it to Google and Bing?
Technical SEO | | EcommerceSite0 -
Linking root domains and youtube
All of my competitors have high linking root domains from youtube and our isn't showing up although we have 1.5 million views to youtube. I tried adding our URL to the videos but it hasn't recognized as a linking root domain. What should I do?? There's a ton of SEO juice here I want to tap into! watch?v=GTXFRTY4CCA&list=UUOcfF9LAHKedNSyk-gk5xDw&index=28
Technical SEO | | tonymartin0 -
Redirecting domain to the main domain (hosting cost?)
Hello Everyone, I have the following situation. There is main domain and a secondary domain that is related to the page on the main domain. I want to integrate the content of the secondary domain into the page on the main domain and redirect the secondary domain via 301 to that specific page. As i understand I can do it via .htaccess using rewrite mechanism. http://www.seomoz.org/learn-seo/redirection But the question is does it mean I have to keep paying for the hosting for the secondary domain? Because htaccess has to be located on the web server so I would need a hosting plan for it? Is that true? Is there any way around it? P.S. to avoid any confusion - I am talking about hosting plan - not domain registration fees
Technical SEO | | SirMax0 -
Any way around buying hosting for an old domain to 301 redirect to a new domain?
Howdy. I have just read this QA thread, so I think I have my answer. But I'm going to ask anyway! Basically DomainA.com is being retired, and DomainB.com is going to be launched. We're going to have to redirect numerous URLs from DomainA.com to DomainB.com. I think the way to go about this is to continue paying for hosting for DomainA.com, serving a .htaccess from that hosting account, and then hosting DomainB.com separately. Anybody know of a way to avoid paying for hosting a .htaccess file on DomainA.com? Thanks!
Technical SEO | | SamTurri0 -
Links from Youtube Channel
I stumbled across this blog post: http://garyreid.com/youtube-removes-nofollow/ and also this one : http://www.kevin-barnes.com/youtube-secret-authority-loophole/ which talks about no-follow links from your Youtube Channel Page. We've setup a Youtube channel, and have begun updating it regularly, however the link appears to be a redirect-type link -presumably this means no link juice is passed? The code of the link on our Youtube channel: http://www.pretavoir.co.uk The second blog mentions building PA on your Youtube channel by commenting on other videos which then links back to your channel page - if that juice can't go to your site, then I assume the technique is of limited use? Apart from boosting your Youtube Channel's rankings of course, which I guess can't hurt.
Technical SEO | | seanmccauley0