Effect duration of robots.txt file.
-
in my web site there is demo site in that also, index in Google but no need it now.so i have created robots file and upload to server yesterday.in the demo folder there are some html files,and i wanna remove all these in demo file from Google.but still in web master tools it showing
User-agent: *
Disallow: /demo/How long this will take to remove from Google ?
And are there any alternative way doing that ?
-
Google Webmaster Tools also has a remove URL function where you can remove an entire directory, which may be of help to you.
-
And, if they are already indexed, you have to wait for them to be recrawled, then fall out of index, so it's not an immediate thing. Sometimes it takes days, sometimes weeks.
-
Hello,
The robots directive will only prevent google from crawling the pages. In order t remove the pages from index you need to add "meta noindex" to the pages you want to have removed.
<meta name="robots" content="noindex">
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93710
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Robots.txt Allowed
Hello all, We want to block something that has the following at the end: http://www.domain.com/category/product/some+demo+-text-+example--writing+here So I was wondering if doing: /*example--writing+here would work?
Intermediate & Advanced SEO | | ThomasHarvey0 -
Will obfuscating HTML have a bad effect on my ranking?
I would like to obfuscate my HTML so that people do not see that I used a Template on my site. Does obfuscating HTML have a bad effect on the ranking in google? Thanks!
Intermediate & Advanced SEO | | RWW0 -
Links from swf file widely distributed?
Hello, I just realised that Google is listing as backlinks, links from swf games that we created and distributed widely. We never used this method to have backlinks but as we create the games and give them for free to other sites, we added a link back to our site, if the user who played the game want to visit us. But I am worried that this is interpreted as black hat seo, and this affected our ranking badly. Anyone had this kind of issue? How do you think we should be tackling this? Is this could be affected our site? Thanks for your help on this guys 😉
Intermediate & Advanced SEO | | drimlike0 -
How to leverage browser cache a specific file
Hello all,
Intermediate & Advanced SEO | | asbchris
I am trying to figure out how to add leverage browser caching to these items. http://maps.googleapis.com/maps/api/js?v=3.exp&sensor=false&language=en http://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js http:///analytics.js Whats hard is I understand the purpose, but unlike a css file, how do you specify an expiration on an actual direct path file? Any help or link to get help is appreciated. Chris0 -
Duplicate Content From Indexing of non- File Extension Page
Google somehow has indexed a page of mine without the .html extension. so they indexed www.samplepage.com/page, so I am showing duplicate content because Google also see's www.samplepage.com/page.html How can I force google or bing or whoever to only index and see the page including the .html extension? I know people are saying not to use the file extension on pages, but I want to, so please anybody...HELP!!!
Intermediate & Advanced SEO | | WebbyNabler0 -
Template Files .tpl versus .html files
We sell a large selection of Insulation Products use template files (.tpl) to collect up-to-date information from a server side database file that contains some 2,500 line items. When an HTML (.html) file is requested on the Internet, the 'example.tpl' file is accessed, the latest product and and pricing information is accessed, then presented to the viewer as 'example.html' My question: Can the use of .tpl files negatively impact Search Engine acceptance?
Intermediate & Advanced SEO | | Collie0 -
Hyphen domain effect SEO?
Hi Guys, I am looking to buy some domain that have the keyword I want in - but my question is; Does using hypehns in a domain effect your SEO? Thanks Gareth
Intermediate & Advanced SEO | | GAZ090 -
Are there any benefits to having dashes in file names?
Through searching, I can find lots of discussion regarding "dash vs underscore", but am having trouble with an even simpler question: Is there any SEO difference between using http://www.broadway.com/shows/milliondollarquartet.php vs. http://www.broadway.com/shows/million-dollar-quartet.php
Intermediate & Advanced SEO | | RyanWhitney150