Effect duration of robots.txt file.
-
in my web site there is demo site in that also, index in Google but no need it now.so i have created robots file and upload to server yesterday.in the demo folder there are some html files,and i wanna remove all these in demo file from Google.but still in web master tools it showing
User-agent: *
Disallow: /demo/How long this will take to remove from Google ?
And are there any alternative way doing that ?
-
Google Webmaster Tools also has a remove URL function where you can remove an entire directory, which may be of help to you.
-
And, if they are already indexed, you have to wait for them to be recrawled, then fall out of index, so it's not an immediate thing. Sometimes it takes days, sometimes weeks.
-
Hello,
The robots directive will only prevent google from crawling the pages. In order t remove the pages from index you need to add "meta noindex" to the pages you want to have removed.
<meta name="robots" content="noindex">
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93710
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Gallery maintenance and the effect on SEO
Basically we get a lot of users uploading photos as part of their review, but many photos aren't moderated into our pages and therefore are never displayed. Things like selfies rather than photos of the product or just random google images that are completely unrelated to our products or services. Is there any benefit in cleaning up the gallery since some images we don't use are just sat there in admin?
Intermediate & Advanced SEO | | Fubra
when a page loads, would it be quicker if we had less content in the gallery? With our SEO hat on.
or does it not matter since it's not loading that content (photos) anyway?0 -
What is the recommended way to save Image Files in WP?
Hi, Is there a recommended setting (or even plugin) to use when saving image files on my Wordpress blog?
Intermediate & Advanced SEO | | BeytzNet
Which folders etc. Thanks0 -
Moving Entire Domain to New Site with New File Extensions
I have been looking for a while for a good an clear Step by Step guide for moving a site from an old to a new domain... so I guess a good discussion here, could help many web masters have a smooth transition. So in your opinion, beside the obvious, what are the most important steps you must take? Here is what I do: 1. 301 old site to new one and TEST.
Intermediate & Advanced SEO | | dhidalgo1
2. Check Internal Links - Double Check for 404's.
3. Update your Social Profiles with new URL.
4. Let GWT and BWT of the change and request a Crawl.
5. Contact as Many of Webmaster as you possibly can to point your links to your new domain. What's missing? What have you found helpful and/or Effective?0 -
Help with Effective Meta Description
Can someone give me a sample of an effective meta description tag? All of the best practice stuff I read doesn't talk about how to raise CTR. It seems to me that this is a neglected area of SEO, and we want to do this right. Obviously, we will need to test. For example, my main home page keyword is "IT Support" Things I might want to put in the tag: Free Network Assessments 100% Risk Free Trials "Relentless IT Support" (Major Theme) 30 Years of Experience (since 1984) Eliminate your IT Support Headaches Forever (Too long) Call to action? BTW, Thanks to everyone for your help. This is a great community. Solid advice from experts. Here's an example of what I would create Relentless IT Support Since 1984. Trust and Accountability. 100% Risk Free Trials. Contact us today for Free IT Assessment.
Intermediate & Advanced SEO | | CsmBill0 -
About robots.txt for resolve Duplicate content
I have a trouble with Duplicate content and title, i try to many way to resolve them but because of the web code so i am still in problem. I decide to use robots.txt to block contents that are duplicate. The first Question: How do i use command in robots.txt to block all of URL like this: http://vietnamfoodtour.com/foodcourses/Cooking-School/
Intermediate & Advanced SEO | | magician
http://vietnamfoodtour.com/foodcourses/Cooking-Class/ ....... User-agent: * Disallow: /foodcourses ( Is that right? ) And the parameter URL: h
ttp://vietnamfoodtour.com/?mod=vietnamfood&page=2
http://vietnamfoodtour.com/?mod=vietnamfood&page=3
http://vietnamfoodtour.com/?mod=vietnamfood&page=4 User-agent: * Disallow: /?mod=vietnamfood ( Is that right? i have folder contain module, could i use: disallow:/module/*) The 2nd question is: Which is the priority " robots.txt" or " meta robot"? If i use robots.txt to block URL, but in that URL my meta robot is "index, follow"0 -
New server update + wrong robots.txt = lost SERP rankings
Over the weekend, we updated our store to a new server. Before the switch, we had a robots.txt file on the new server that disallowed its contents from being indexed (we didn't want duplicate pages from both old and new servers). When we finally made the switch, we somehow forgot to remove that robots.txt file, so the new pages weren't indexed. We quickly put our good robots.txt in place, and we submitted a request for a re-crawl of the site. The problem is that many of our search rankings have changed. We were ranking #2 for some keywords, and now we're not showing up at all. Is there anything we can do? Google Webmaster Tools says that the next crawl could take up to weeks! Any suggestions will be much appreciated.
Intermediate & Advanced SEO | | 9Studios0 -
What is the effect on using jQuery sliders for content on SEO?
I know using css in subversive manners gets you dinged for points. I didnt know if JS counted the same since you are essentially hiding parts of the content and showing it in intervals as slides. The goal would be having key items for a client in divs and rotating those divs via a slider plugin as slides. I was just curious if that effected things in any way. Thanks! ~Paul
Intermediate & Advanced SEO | | peb72680 -
Would using display:none; to hide a section of text effect SEO negatively?
I have seen several sites that put a div feature at the bottom of a page to hide content. If you click on the button, it will extend the page down and be loaded with paragraphs of text rich with keywords. Does anyone know is this is viewed as a negative with Google?
Intermediate & Advanced SEO | | netmkting0