Site blocked by robots.txt and 301 redirected still in SERPs
-
I have a vanity URL domain that 301 redirects to my main site. That domain does have a robots.txt to disallow the entire site as well.
However, for a branded enough search that vanity domain still shows up in SERPs and has the new Google message of:
A description for this result is not available because of this site's robots.txt
I get why the message is there - that's not my , my question is shouldn't a 301 redirect trump this domain showing in SERPs, ever?
Client isn't happy about it showing at all.
How can I get the vanity domain out of the SERPs?
THANKS in advance!
-
Add both sites to GWT, then go to: Vanity Site > Configuration > Change of Address > Select the new domain
Give it some time and G should replace the vanity url to the new domain.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Good robots txt for magento
Dear Communtiy, I am trying to improve the SEO ratings for my website www.rijwielcashencarry.nl (magento). My next step will be implementing robots txt to exclude some crawling pages.
Technical SEO | | rijwielcashencarry040
Does anybody have a good magento robots txt for me? And what need i copy exactly? Thanks everybody! Greetings, Bob0 -
Robots.txt Download vs Cache
We made an update to the Robots.txt file this morning after the initial download of the robots.txt file. I then submitted the page through Fetch as Google bot to get the changes in asap. The cache time stamp on the page now shows Sep 27, 2013 15:35:28 GMT. I believe that would put the cache time stamp at about 6 hours ago. However the Blocked URLs tab in Google WMT shows the robots.txt last downloaded at 14 hours ago - and therefore it's showing the old file. This leads me to believe for the Robots.txt the cache date and the download time are independent. Is there anyway to get Google to recognize the new file other than waiting this out??
Technical SEO | | Rich_A0 -
"Extremely high number of URLs" warning for robots.txt blocked pages
I have a section of my site that is exclusively for tracking redirects for paid ads. All URLs under this path do a 302 redirect through our ad tracking system: http://www.mysite.com/trackingredirect/blue-widgets?ad_id=1234567 --302--> http://www.mysite.com/blue-widgets This path of the site is blocked by our robots.txt, and none of the pages show up for a site: search. User-agent: * Disallow: /trackingredirect However, I keep receiving messages in Google Webmaster Tools about an "extremely high number of URLs", and the URLs listed are in my redirect directory, which is ostensibly not indexed. If not by robots.txt, how can I keep Googlebot from wasting crawl time on these millions of /trackingredirect/ links?
Technical SEO | | EhrenReilly0 -
301 redirecting old content from one site to updated content on a different site
I have a client with two websites. Here are some details, sorry I can't be more specific! Their older site -- specific to one product -- has a very high DA and about 75K visits per month, 80% of which comes from search engines. Their newer site -- focused generally on the brand -- is their top priority. The content here is much better. The vast majority of visits are from referrals (mainly social channels and an email newsletter) and direct traffic. Search traffic is relatively low though. I really want to boost search traffic to site #2. And I'd like to piggy back off some of the search traffic from site #1. Here's my question: If a particular article on site #1 (that ranks very well) needs to be updated, what's the risk/reward of updating the content on site #2 instead and 301 redirecting the original post to the newer post on site #2? Part 2: There are dozens of posts on site #1 that can be improved and updated. Is there an extra risk (or diminishing returns) associated with doing this across many posts? Hope this makes sense. Thanks for your help!
Technical SEO | | djreich0 -
I accidentally blocked Google with Robots.txt. What next?
Last week I uploaded my site and forgot to remove the robots.txt file with this text: User-agent: * Disallow: / I dropped from page 11 on my main keywords to past page 50. I caught it 2-3 days later and have now fixed it. I re-imported my site map with Webmaster Tools and I also did a Fetch as Google through Webmaster Tools. I tweeted out my URL to hopefully get Google to crawl it faster too. Webmaster Tools no longer says that the site is experiencing outages, but when I look at my blocked URLs it still says 249 are blocked. That's actually gone up since I made the fix. In the Google search results, it still no longer has my page title and the description still says "A description for this result is not available because of this site's robots.txt – learn more." How will this affect me long-term? When will I recover my rankings? Is there anything else I can do? Thanks for your input! www.decalsforthewall.com
Technical SEO | | Webmaster1230 -
Robots.txt not working?
Hello This is my robots.txt file http://www.theprinterdepo.com/Robots.txt However I have 8000 warnings on my dashboard like this:4 What am I missing on the file¿ Crawl Diagnostics Report On-Page Properties <dl> <dt>Title</dt> <dd>Not present/empty</dd> <dt>Meta Description</dt> <dd>Not present/empty</dd> <dt>Meta Robots</dt> <dd>Not present/empty</dd> <dt>Meta Refresh</dt> <dd>Not present/empty</dd> </dl> URL: http://www.theprinterdepo.com/catalog/product_compare/add/product/100/uenc/aHR0cDovL3d3dy50aGVwcmludGVyZGVwby5jb20vaHAtbWFpbnRlbmFjZS1raXQtZm9yLTQtbGo0LWxqNS1mb3ItZXhjaGFuZ2UtcmVmdWJpc2hlZA,,/ 0 Errors No errors found! 1 Warning 302 (Temporary Redirect) Found about 5 hours ago <a class="more">Read More</a>
Technical SEO | | levalencia10 -
Google Confusion: Two Sites and a 301 Redirect.
Hi, We have a client who just sprang a new project on us. As always, they went ahead and did some stuff before bringing us into the loop! (oh the joy of providing SEO services!) Anyway, i'm pretty swamped right now and need some extra brains on this. Basically the client had www.examplesiteA.com online for many years (an affiliate site which had built up a strong brand in the industry). They have now decided to turn this affiliate site into a full blown service platform and so with the new site being built they 301'd the whole thing over to www.examplesiteB.com - this is where they want all the old affiliate content to be hosted. So essentially examplesiteA.com is now examplesiteB.com and a new site is being placed on examplesiteA.com - still with me? So this has all happened and a brand new website is on examplesiteA.com and the old examplesiteA is now sitting exactly as it used to, but on the examplesiteB domain. The 301 redirect has been removed and the new examplesiteA seems to have been crawled, but the homepage is not indexed. When you search for examplesiteA, examplesiteB is the top result. Now they are similar domain names and to be fair I have very little data at this point i.e. I don't know when the 301 redirect was removed and it maybe that this all fixes itself with time. How is link equity effected now that examplesiteA.com was 301 redirected to examplesiteB.com and cached in this way, but now the 301 redirect has been removed and does not exist? Would link juice have been diluted throughout the process? Obviously if we had been in on all this before anything was implemented we would have done things differently. Interested to hear what others would do coming in at this point. Thanks and look forward to the advice!
Technical SEO | | MarcLevy0 -
301 Redirect Issue
I'm having an issue with 301 redirects: Let's see if I can verbalize my thoughts on this one... So we just recently moved our site to Wordpress. One of our new 301 commands is redirecting oursite.com/news to oursite.com/blog . However there are other links from our previous site that look like oursite.com/news/XYZ and the issue is that, because wordpress structures its links differently, that URL is not equivalent to oursite.com/blog/XYZ. Instead, it might look something more like oursite.com/blog/yaddayadda/XYZ or something. Does that make sense? The issue is that when I find an old link of ours on google that looks something like "oursite.com/news/XYZ" or "oursite.com/news/ABC" it is automatically replacing "news" with "blog". When I try to go in manually and redirect anything that says "/news/XYZ" to "/blog/yaddayadda/XYZ" it still doesn't work. It still just replaces "news" with "blog." Wow I realize that might not make sense to anyone but if it does - please advise!! Thanks!!!!
Technical SEO | | EntrustSEO0