Adding the link masking directory to robots.txt?
-
Hey guys,
Just want to know if you have any experience with this.
Is it worthwhile blocking search engines from following the link masking directory.. (what i mean by this is the directory that holds the link redirectors to an affiliate site:
example:
mydomain.com/go/thislinkgoes to
amazon.com/affiliatelinkI want to know if blocking the 'go' directory from getting crawled in robots.txt is a good idea or a bad idea?
I am not using wordpress but rather a custom built php site where i need to manually decide on these things.
i want to specifically know if this in any way violates guidelines for google. it doesn't change the custom experience because they know exactly where they will end up if they click on the link.
any advice would be much appreciated.
-
Iredeto,
I do this on a few of my sites and it works out well. Saves on crawl budget, keeps google from accessing my affiliate links, and keeps any pagerank from passing through the links, which keeps me in-line with Google's webmaster policy on links.
One thing to keep in mind is that Google may show those URLs in the search results with a message that they can't show the content because it has been blocked. If you want to keep this from happening you may have to remove that directory GWT using the URL Removal tool. If they get re-indexed in 90 days (or whatever the reset time frame is) you will have to do it again. Hopefully that won't be an issue once you get them all removed the first time and block the folder.
Using rel = "nofollow" tags on the hyperlinks going to that directory wouldn't hurt either.
-
Hi!
I know cases where you can hide the contents of a folder to Google to not affect the rest of the website.
Note that you are facing a machine that knows how to crawl and index content.
Also be sure to block with noindex nofollow meta tags and removing URLs from Google index if at some point you had not locked.
Best regards!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Adding / Manipulating Page Meta Titles?
We have a client who is experiencing some heavy google modification to the title tags being displayed on the search engine. It is adding "- 0 Reviews" to an ecommerce site. Obviously a bad start. There were no instances of these keywords anywhere on any of these pages, header tag or otherwise (on only a handful of the affected pages there was a single commented out image with an alt tag 0 reviews - but it was commented out and since removed) We have attempted to rewrite the title multiple times and it will modify the title but still include the non-relevant addition. Has anyone ever experienced anything like this?
Algorithm Updates | | Spindle0 -
Hi guys, I have a question about linking to a product page for linkbuilding. Does that count adversely vs. linking to a homepage?
Hi - so until now we have been building links via blog posts and articles and linking them to the homepage. It seems the ranking of some of my top keywords has fallen so had a few questions/concerns: Does it affect the rankings adversely if I link to the product page vs the homepage? What is rule of thumb for increasing rankings of inside pages/keywords and building links to them? Thanks
Algorithm Updates | | DGM0 -
Links from high Domain authority sites
I have a relatively uncompetitive niche ranking around number 6 for my keywords. Would getting a few links from some Moz DA 80-90 and DA 90-100 sites help my rankings a lot? Some of the pages linking to me from these sites might be deep in the site pretty far away from the home page with pagerank of "unranked" or a grayed out bar and these pages linking to me might not have many links at all other than from the internal links of the site itself and would have a Moz PA of 10 or 20. Would these pass much pagerank or authority to my site or would they not be worth going after? These links to my site would be in context on a blog. Thanks mozzers!
Algorithm Updates | | Ron100 -
How to content marketing: Should my blog posts link to my sales page?
Hi, I've been doing a weekly blog making sure that each blog post contains my money keywords in the text, sometimes in h2 tags etc. My blog posts never contain any links to my actual sales page. Should I link each blog post to my sale page or is it overdoing it? Will internal linking of all my blog posts to my sales page will improve its page authority or have any SEO benefits? What about using exact match anchor text on these internal links? I couldn't find any resource online about this matter. Thank you for your opinion and help! -Marc
Algorithm Updates | | marcandre0 -
Member's Badge as Link Building to Homepage or Internal Pages?
Providing members and embeddable badge is a well known link building tactic. Is it better to have the badges from hundreds or even thousands of members link back to the homepage of a website, or a lot of different inner pages? The inner pages would the their individual's profile which sits under a category (such as a service and organisation by location). Member's websites would be related to the content of the website generally. What are the advantages of each? 1. Links to homepage make it easier to rank for competitive keywords on the homepage? If the types of websites were to vary a lot, say a carpet cleaning website and a web designer website, if they all linked to the homepage, would it cause some confusion for the link profile?
Algorithm Updates | | designquotes0 -
Loss of 1,000 links has negatively affecting rankings
Hey there, One of the clients we're working with has lost about 1,000 or so backlinks over the last two or three months - mainly old article and directory links - and it has massively affected the site's search rankings. The site was ranking for pretty much all of its keywords in prominent positions on Google (mostly first page) but has now seen positions dive to 50, 90 and even outside of the top 100. Is there any immediate action we can put into place to help restore our rankings?
Algorithm Updates | | Webrevolve0 -
Does the page title keyword count in anchor text when link is web address?
If someone links to my plumbing site with this link as the anchor text: http://www.plumbers.com/austin-plumbers.html does the key phrase "austin plumbers" get counted in the anchor text by google or is this a sample of anchor text that google ignores? Thanks mozzers! Ron
Algorithm Updates | | Ron100 -
Has Google lost its mind? I am the only link in every SERP for a query?
I run a small online debutante dress store and have been doing some onsite seo recently. Anyways, when I search for the search query "deb dress style guide" my site is the only search result for the first three pages of Google Australia (my target market). Just endless links to my site. I have competitors in my niche who all have websites worthy of listing in the SERP as shown when you google "deb dresses". Can anyone explain whats going on?
Algorithm Updates | | mydebdress20