External Links from own domain
-
Hi all,
I have a very weird question about external links to our site from our own domain.
According to GWMT we have 603,404,378 links from our own domain to our domain (see screen 1) We noticed when we drilled down that this is from disabled sub-domains like m.jump.co.za.
In the past we used to redirect all traffic from sub-domains to our primary www domain. But it seems that for some time in the past that google had access to crawl some of our sub-domains, but in december 2010 we fixed this so that all sub-domain traffic redirects (301) to our primary domain. Example http://m.jump.co.za/search/ipod/ redirected to http://www.jump.co.za/search/ipod/
The weird part is that the number of external links kept on growing and is now sitting on a massive number.
On 8 April 2011 we took a different approach and we created a landing page for m.jump.co.za and all other requests generated 404 errors. We added all the directories to the robots.txt and we also manually removed all the directories from GWMT.
Now 3 weeks later, and the number of external links just keeps on growing: Here is some stats:
11-Apr-11 - 543 747 534
12-Apr-11 - 554 066 716
13-Apr-11 - 554 066 716
14-Apr-11 - 554 066 716
15-Apr-11 - 521 528 014
16-Apr-11 - 515 098 895
17-Apr-11 - 515 098 895
18-Apr-11 - 515 098 895
19-Apr-11 - 520 404 181
20-Apr-11 - 520 404 181
21-Apr-11 - 520 404 181
26-Apr-11 - 520 404 181
27-Apr-11 - 520 404 181
28-Apr-11 - 603 404 378
I am now thinking of cleaning the robots.txt and re-including all the excluded directories from GWMT and to see if google will be able to get rid of all these links.
What do you think is the best solution to get rid of all these invalid pages.
-
We had 301s for about 6 months, and the old URLs did not disappear from google. Thats why we decided to change them to 404s, with the thinking that Google might remove them quicker. But the number of links from sub-domains just keeps on growing.
I am worried that by having these problem urls listed in the robots.txt actually prevents google from following them and seeing that it should be removed and that it returns a 404
-
Instead of trying to manage a massive 301 list, can you just customize your 404 page to redirect?
{script to test page URL}
$location = "http://www.YourSite.com/";
header("HTTP/1.1 301 Moved Permanently");
header("Location: {$location}");
exit;
}
-
Update:
There are 2 things that still puzzles me with this:
If you go to http://www.google.co.za/search?q=site:jump.co.za+-www&hl=en&rlz=1C1GPCK_enZA426ZA426&prmd=ivns&filter=0&biw=1920&bih=979 you notice all sorts of weird sub-domains, and all of these are invalid and have been removed from GWMT.
If you manage the domain m.jump.co.za on GWMT you also notice that it still reports on keywords, queries and all sorts of data, although the site is disabled and all the URLs generate 404 errors
There is only a few of these weird sub-domains that are causing the problems:
0www.
iiiiiwww.
iwww.
m.
wtfwww.
www.www.
wwww.All these domains feels very fimiliar to me and I am almost 100% sure that its domains that used to test when we found the problem on apache, meaning google took the data from the toolbar queries and probably started indexing these sub-domains. But now I can't get rid of them, and Google seems to be out of control with these.
So the main question is probably, should we just give 404s or should we add to Robots.txt as well?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Will links be counted?
We are considering a redesign of our website and one of the options we are considering is to come up with something along the lines of http://www.tesco.com/, with rotating top offers. The question I am wondering is whether or not the links (ie. the blue links on the left side of the main graphic) will be visible to the spiders, and if not, whether there is a way to code it so they are?
Technical SEO | | simonukss0 -
Disavowing links, Is it effective?
Looking for your experiences with disavowing back-links? We've been flooded with new clients who need spammy link removal services and wanted to hear more about your experience with the disavow tool. For sites that have been penalized, how long did it take for them to come back using the disavow tool? Did you see sites come back after the next algo update? Here's the basics of our services for link deletion: 1. Find all the spammy links
Technical SEO | | Keith-Eneix
2. Contact webmasters to delete them
3. Disavow all spammy links that are part of an obvious network
4. Implement a content plan for new quality links to get the site healthy again.
5. Report on all links removed and new links attained Just want to make sure our processes are in line with what everyone else is doing?0 -
Feefo review links
Hi guys, so we are on feefo and noticed links coming in per review for different anchor text, this will be done on mass due to the amount of reviews we will get - this is all natural but in SEO site-wide links are typically not. How if at all do you think Google will react to this?
Technical SEO | | pauledwards0 -
Google Links
I am assuming that the list presented by Google Webmaster tools (TRAFFIC | Links To Your Site) is the one that will actually be used by Google for indexing ? There seem to be quite a few links that there that should not be there. ie Assumed NOFOLLOW links. Am I working under an incorrect assumption that all links in webmaster tools are actually followed ?
Technical SEO | | blinkybill0 -
Google Shows 24K Links b/w 2 sites that are not linked
Good Morning, Does anyone have any idea why Google WMT shows me that i have 24,101 backlinks from one of my sites ( http://goo.gl/Jb4ng ) pointing to my other site ( http://goo.gl/JgK1e ) ... These sites have zero links between them, as far as I can see/tell. Can someone please help me figure out why Google is showing 24k backlinks? Thanks
Technical SEO | | Prime850 -
Parking Domains
I currently have a website domain.com.au, an American branch of the company who own domain.com are currently having their site built and want to forward there domain.com to domain.com.au while construction is taking place. Are there any negative effects to parking the domain.com on my domain.com.au? What is the best method to do this without causing any problems for my domain.com.au?
Technical SEO | | Pork0 -
Old owners links pointing to new owners domain
We have a number of web sites. We recently acquired an excellent domain name, it happened to be owned by one of our competitors. Our competitor has a lot of web sites, each domain having a basic 5 page unoptimized site with one of those pages little more than a link farm. They have over three hundred domains with all but lets say 10 of them consisting of basic 5 page sites with a link "directory" on one of those pages. - the directory page is the same on every single site/domain. One of the links from that directory is going to our newly acquired domain and newly optimized web site. Being new to this, should this pose any kind of concern for us? Thanks in advance!
Technical SEO | | PlasticCards0 -
Keyword rich domains
Hi, Our site is beingthere.com.au We are in the business of video conferencing in Australia. I was wondering if there would be any benefit of purchasing keyword rich domains such as www.videoconferencing.net.au www.video streaming.net.au What would be the benefit(s)? And How would I go about using these domains to maximise SEO benefit? Thanks Dan
Technical SEO | | dantmurphy0