Google is indexing my directories
-
I'm sure this has been asked before, but I was looking at all of Google's results for my site and I found dozens of results for directories such as:
Index of /scouting/blog/wp-includes/js/swfupload/plugins
Obviously I don't want those indexed. How do I prevent Google from indexing those?
Also, it only seems to be doing it with Wordpress, not any of the directories on my main site. (We have a wordpress blog, which is only a portion of the site)
-
Thanks for expanding on that!
a) There are no pages in these folders; they're just directories full of Wordpress config stuff
b) I already have Yoast and have tag pages noindexed. All the directories I'm referring to are similar to the path I showed above.
c) If I request removal of specific folders, will it also remove all the files within the folder?\
Thanks!
-
To elaborate a little on what he said
a) yes block the folders with content you don't want indexed in robots.txt - but that is not enough you need to add the noindex to the pages as well for an extra measure.
b) get the YOAST seo plugin for wordpress installed on your site and configure it to follow,noindex these page. look at other sections you don't want indexed too like tag pages. This plugin is easy to use and will help suggest default settings.
c) yes you can go into WMT to request removal of specific folders and urls and Google will quickly get them out of the index for you instead of waiting for them to age away. Your robots.txt should be in place before doing this to make sure it's permanent.
-
Here are your options:
- Robots.txt - http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93708 - easy option.
- Noindex tags - http://www.mattcutts.com/blog/google-noindex-behavior/
- Removal request - https://support.google.com/webmasters/bin/answer.py?hl=en&answer=59819
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why is my site not being indexed?
Hi, I have performed a site:www.menshealthanswers.co.uk search on Google and none of the pages are being indexed. I do not have a "noindex" value on my robot tag This is what is in place: Any ideas? Jason
Technical SEO | | Jason_Marsh1230 -
My whole directory dropped from google
Hi there I have lost a whole directory in Google rankings and this is where my traffic was coming from. Do you know why this would happen? It is the eCommerce part of the site. So if I do site:www.domain.com the site is there and then site:www.domain.com/directory is not there any more. Thanks for your help.
Technical SEO | | vcasebourne0 -
Google Places, Google Plus, Oh my!
Ok - So I am in the position to try and clean up the current Google places nightmare for a company. Right now there is about 3 or 4 different google places listings for them that they have no control over. So here is what I did: 1. I took control of them all by verifying via phone and confirmed all of them. 2. I suspended all the listings but 1 3. I edited the one listing to be accurate and complete.
Technical SEO | | DylanPKI
Then I waited, and waited... A month later, the old listings are still up and none of the changes to the one listing have been made. Today it gets a bit more complicated. Today I created a Google+ page for the business which seems like it may end up adding yet ANOTHER Google Places listing, is that correct? They are sending a post card to verify, but I have the page all set up ready to go and plan on tying it to the website. I am not exactly sure what my specific question is, but I am looking for any advice anyone has on the best way to go about this situation. Thank you in advance!0 -
Is Google caching date same as crawling/indexing date?
If a site is cached on say 9 oct 2012 doesn't that also mean that Google crawled it on same date ? And indexed it on same date?
Technical SEO | | Personnel_Concept0 -
How to remove entire directory off Google's Cache
The old version of Webmaster tools used to allow you to select whether to remove a single page from index or an entire directory.
Technical SEO | | vpahwa
http://www.canig.com/pageimages/submitremovalrequest.jpg How can I do this with the new Webmaster Tools? I can't find the option to remove an entire directory.0 -
Problem with indexing
Hello, we've changed our CMS recently, everything seems to work well, but for some reason google, and other crawlers can't see or index other pages than main. There is no restriction in robots, nor any other visible issue. Please help if you can. Website: http://www.design-glassware.com/
Technical SEO | | divan0 -
Is this against google rules
Hi i am wanting to know if this is against google rules. I am building a website which will have lots of different sections and i wanted to know if you were allowed to have a new domain name pointing to a section of the site. so for example if i had a site with a domain name of manchester and then i wanted a section of the site to be called www.manchester.com/complimentary health I want to know if to help with traffic to the site and to have a better domain name, if it was allowed to have a new domain name pointing to that section of the site which could be called www.complimentaryhealth.com and have that pointing to the section. would love to hear your thoughts on this
Technical SEO | | ClaireH-1848860 -
How long does it take for Google to de-index urls?
Added the noindex meta tag to some pages on my site and I am wondering if anyone has any idea how long it will take to deindex the urls?
Technical SEO | | nicole.healthline0