Removal request for entire catalog. Can be done without blocking in robots?
-
Bunch of thin content (catalog) pages modified with "follow, noindex" few weeks ago. Site completely re-crawled and related cache shows that these pages were not indexed again. So it's good I suppose
But all of them are still in main Google index and shows up from time to time in SERPs. Will they eventually disappear or we need to submit removal request?Problem is we really don't want to add this pages into robots.txt (they are passing link juice down below to product pages)Thanks!
-
If your intention is to keep the page for "follow" I would not submit a removal request for the page since that will wipe it off the map. Google needs to crawl your page to know you don't want it indexed.
As I said, Google already re-crawled them and they are not re-indexed. Last cache dates show all is correctly lef out of new indexation.
I guess my concern was how long this pages will stay in index. That was done to make sure Panda will be able to recalculate the value of the site.
-
If your intention is to keep the page for "follow" I would not submit a removal request for the page since that will wipe it off the map. Google needs to crawl your page to know you don't want it indexed.
Things can take a while, sometimes many months before its not indexed properly. Google also has a disclaimer that it could possibly keep the page indexed for a while.
Also you have to make sure that you do some post things after the removal is made to ensure it never comes back
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1663419
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Adding directories to robots nofollow cause pages to have Blocked Resources
In order to eliminate duplicate/missing title tag errors for a directory (and sub-directories) under www that contain our third-party chat scripts, I added the parent directory to the robots disallow list. We are now receiving a blocked resource error (in Webmaster Tools) on all of the pages that have a link to a javascript (for live chat) in the parent directory. My host is suggesting that the warning is only a notice and we can leave things as is without worrying about the page being de-ranked/penalized. I am wondering if this is true or if we should remove the one directory that contains the js from the robots file and find another way to resolve the duplicate title tags?
Technical SEO | | miamiman1000 -
Block bad crawlers
Hi! how are you? I've been working on some of my sites, and noticed that i'm getting lots of crawls by search engines that i'm not intereted in ranking well. My question is the following: do you have a list of 'bad behaved' search engines that take lots of bandwidth and don´t send much/good traffic? If so, do you know how to block them using robots.txt? Thanks for the help! Best wishes, Ariel
Technical SEO | | arielbortz0 -
Robots.txt Question
In the past, I had blocked a section of my site (i.e. domain.com/store/) by placing the following in my robots.txt file: "Disallow: /store/" Now, I would like the store to be indexed and included in the search results. I have removed the "Disallow: /store/" from the robots.txt file, but approximately one week later a Google search for the URL produces the following meta description in the search results: "A description for this result is not available because of this site's robots.txt – learn more" Is there anything else I need to do to speed up the process of getting this section of the site indexed?
Technical SEO | | davidangotti0 -
Should I block robots from URLs containing query strings?
I'm about to block off all URLs that have a query string using robots.txt. They're mostly URLs with coremetrics tags and other referrer info. I figured that search engines don't need to see these as they're always better off with the original URL. Might there be any downside to this that I need to consider? Appreciate your help / experiences on this one. Thanks Jenni
Technical SEO | | ShearingsGroup0 -
Do I need robots.txt and meta robots?
If I can manage to tell crawlers what I do and don't want them to crawl for my whole site via my robots.txt file, do I still need meta robots instructions?
Technical SEO | | Nola5040 -
How can I see if my website was penalize by Google?
Hello, I have a website http://digitaldiscovery.eu that I have been working for 7 months. Everything is alright in the index of the search engines like Google, Bing e Yahoo. I also have like 1000 visits a month wich is not bad for the topic Im pointing at in my country. However my pagerank insist to be on 0, and I really dont understand why. Some of the my competitors that started at the same time, already have a pagerank of 3 and they do not have the same visitors that I do. In the rank system of Alexa im climbing very fast and the visits of my website are growing. So why does the pagerank dont climb aswell?! Tks in advance, Pedro M Pereira
Technical SEO | | PedroM0 -
Linking C Class Blocks Problem
Hi 🙂 I've just discovered that my client, who has a medical practice, has created a series of micro sites about their doctors (around 10 or so). The problem is that they're on a shared host with the same C-class, providing no real link benefit at all. Would it be best to: A) Look for seperate C class hosts for each site & migrate B) Recreate the pages on the main site & 301 all doctor micro sites to new pages C) Leave as is and pursue other link building activites? Has anyone run into a similar issue before? Thanks a bunch! Woj
Technical SEO | | wojkwasi0