Adding the link masking directory to robots.txt?
-
Hey guys,
Just want to know if you have any experience with this.
Is it worthwhile blocking search engines from following the link masking directory.. (what i mean by this is the directory that holds the link redirectors to an affiliate site:
example:
mydomain.com/go/thislinkgoes to
amazon.com/affiliatelinkI want to know if blocking the 'go' directory from getting crawled in robots.txt is a good idea or a bad idea?
I am not using wordpress but rather a custom built php site where i need to manually decide on these things.
i want to specifically know if this in any way violates guidelines for google. it doesn't change the custom experience because they know exactly where they will end up if they click on the link.
any advice would be much appreciated.
-
Iredeto,
I do this on a few of my sites and it works out well. Saves on crawl budget, keeps google from accessing my affiliate links, and keeps any pagerank from passing through the links, which keeps me in-line with Google's webmaster policy on links.
One thing to keep in mind is that Google may show those URLs in the search results with a message that they can't show the content because it has been blocked. If you want to keep this from happening you may have to remove that directory GWT using the URL Removal tool. If they get re-indexed in 90 days (or whatever the reset time frame is) you will have to do it again. Hopefully that won't be an issue once you get them all removed the first time and block the folder.
Using rel = "nofollow" tags on the hyperlinks going to that directory wouldn't hurt either.
-
Hi!
I know cases where you can hide the contents of a folder to Google to not affect the rest of the website.
Note that you are facing a machine that knows how to crawl and index content.
Also be sure to block with noindex nofollow meta tags and removing URLs from Google index if at some point you had not locked.
Best regards!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I control the "link-tree" part of the SERP results?
Hey Mozanarians 🙂 Is there a way to change the pages that are shown on the "link tree"? (e.g. see bellow picture)
Algorithm Updates | | DanielBernhardt
Lets say that I dont want the "career" page to show on the "link tree" can I do that? Thanks and lots of love!
Daniel Bernhardt BLo9KSr.jpg0 -
How much link juice does a sites homepage pass to inner pages and influence inner page rankings?
Hi, I have a question regarding the power of internal links and how much link juice they pass, and how they influence search engine ranking positions. If we take the example of an ecommerce store that sells kites. Scenario 1 It can be assumed that it is easier for the kite ecommerce store to earn links to its homepage from writing great content on its blog, as any blogger that will link to the content will likely use the site name, and homepage as anchor text. So if we follow this through, then it can be assumed that there will eventually be a large number of high quality backlinks pointing to the sites homepage from various high authority blogs that love the content being posted on the sites blog. The question is how much link juice does this homepage pass to the category pages, and from the category pages then to the product pages, and what influence does this have on rankings? I ask because I have seen strong ecommerce sites with very strong DA or domain PR but with no backlinks to the product page/category page that are being ranked in the top 10 of search results often, for the respective category and product pages. It therefore leads me to assume that internal links must have a strong determiner on search rankings... Could it therefore also be assumed that a site with a PR of 5 and no links to a specific product page, would rank higher than a site with a PR of 1 but with 100 links pointing to the specific product page? Assuming they were both trying to rank for the same product keyword, and all other factors were equal. Ie. neither of them built spammy links or over optimised anchor text? Scenario 2 Does internal linking work both ways? Whereas in my above example I spoke about the homepage carrying link juice downward to the inner category and product pages. Can a powerful inner page carry link juice upward to category pages and then the homepage. For example, say the blogger who liked the kite stores blog content piece linked directly to the blog content piece from his site and the kite store blog content piece was hosted on www.xxxxxxx.com/blog/blogcontentpiece As authority links are being built to this blog content piece page from other bloggers linking to it, will it then pass link juice up to the main blog category page, and then the kite sites main homepage? And if there is a link with relevant anchor text as part of the blog content piece will this cause the link juice flowing upwards to be stronger? I know the above is quite winded, but I couldn't find anywhere that explains the power of internal linking on SERP's... Look forward to your replies on this....
Algorithm Updates | | sanj50500 -
Agency footer link, do we keep it ?
Hello ! I was wondering if it's still a good idea to let a do-follow link on the bottom of agency released websites. Because they obvisouly come from different websites with no link with a web marketing agency. Do we have to keep them in the footer in no-follow ? If we do so, how to get some link juice from the different websites ? It sounds a bit stupid but one of my partners went from PR7 to PR5 recently. I guess Penguin 2.0 did not like all its links from its customers' website. Tks a lot !
Algorithm Updates | | AymanH0 -
Diluting your authority - adding pages diluting rankings of other pages?
I'm looking after a site that has around 400 pages. All of these pages rank pretty well for the KW they are targetting. My question is: if we add another 400 pages without doing any link building work, holding DA the same, 1) would the rankings of those 400 previously good pages diminish? and 2) Would the new pages, as more and more new ones are created, rank less and less well?
Algorithm Updates | | xoffie0 -
Need List of new high pr free local USA based directories list.
Need List of new high pr free local USA based directories list. Anyone can help ?
Algorithm Updates | | mnkpso0 -
Loss of 1,000 links has negatively affecting rankings
Hey there, One of the clients we're working with has lost about 1,000 or so backlinks over the last two or three months - mainly old article and directory links - and it has massively affected the site's search rankings. The site was ranking for pretty much all of its keywords in prominent positions on Google (mostly first page) but has now seen positions dive to 50, 90 and even outside of the top 100. Is there any immediate action we can put into place to help restore our rankings?
Algorithm Updates | | Webrevolve0 -
Increase of Directory websites in SERPs
Has anyone noticed a bump for Directory style websites in SERP after the recent Panda/Penguin updates . I am new to SEO but after monitoring one website in particular I have noticed that the site has dropped around 8 positions for location based queries and directory style sites such as yell.com, freeindex.co.uk, thompsonlocal.com all have seemed to moved up ever so slightly even above competior sites, which from memory were all below competior sites in previous search queries . I wonder what your thoughts are on this, or its possible that my site in question has been affected and I need to carry out more work to improve its positions back to its original. Your thoughts are welcome.
Algorithm Updates | | Carden0 -
Too many page links?`
Hi there This blog insert was flag suggesting there was too many page links? I cant identify the same problem? Can anyone explain?
Algorithm Updates | | footballfriends0