How long does Google take to reduce the index size?
-
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
-
How long does it take before it removes most of the duplicated search result pages from the index?
Every site is different but I have seen it take 6 - 9 months for pages to drop out.
Is it still crawling those pages but has not fully decided to remove most of them?
It's possible. As Gaston has already pointed out, search engines will need to access those files again to see you want them noindexed.
How bad is this for SEO?
It temporarily dilutes the amount of SEO equity available to flow to pages you DO want indexed.
-
Hello there,
Did you left some time, without blocking those pages, to google bot to recrawl them?
If you implemented at the same time the noindex tag and the disallow in the robots.txt you are not letting google know that those pages should be deindexed.
Remember that blocking pages in the robots.txt avoid to be scanned again and the new robots tag is not seeng by google bot.My advise is to let google bot recrawl all those pages and wait a few days, may be 2-3 weeks. Slowly the amount of indexed pages will decrease.
Hope i've helped.
Best luck.
GR.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My exactly website name is not in google search
my web site tournet.co but when i search that in google , google can not find it in search result . i have to say my website is a new site and its age is 3 month .
On-Page Optimization | | p.farrhad12365410 -
Does the content in Joomla modules get indexed
Hello Moz, we are using Joomla to build websites. In Joomla you can put content in modules. What we would like to know if would search engines index content in modules, as it does with content attached to menu links? Thanks Ian
On-Page Optimization | | Substance-create0 -
Index Page Content
Mozers, I am of the believe and as a person who puts the utmost emphasis on the index page of any website I am trying to rank, especially with a new domain ... insuring content is relevant, structured, optimized and we have some link juice flowing in. I find once we get the index page ranked, Google's little bots then start to index and rank accordingly the rest of the website ... and we start producing results. We also develop websites (dare I say its where we expertise in) and unexpectantly the client has asked us to carry out SEO work additionally to their web development. Problem lies here, their index page, has absolutely no written content at all, just one large image with a logo (Fashion Website) ...Which I identify as a huge issue as per my explanation is paragraphs one or two. I am sure withe the many more qualified SEO experts and gurus within the SEOmoz community, you have also come across this issue So a few questions, if you don't mind adding advice. 1 - Am I putting too much emphasize on content within the index page, in terms of indexing and actually ranking ...yes I appreciate that terms within the website will be ranked against other pages other than the index page, but will it harm us for having no content at all within the index page 2 - If so, and yes is the answer to above, how do we handle it, we have spoke with the client and he is pretty adamant that he want the index page as is, he has been through out the whole website building process. As suggested, any advice would be really appreciated, its a difficult market to rank within a it is, and i can only see this index page making the task a lot more difficult Cheers John
On-Page Optimization | | Johnny4B0 -
Can Rankings in Google differ so much from computer to computer.
I was telling my friend via facebook to go on my website, I told him to search 'nightlife forum' in google. To which, I believed it was 11th, top of second page. On his computer, its currently ranking at 1st place is it possible to have a difference of 10 places? even though he lives in the same city as me. Would be good to see what it ranks on your computers too google "nightlife forum" look for www.talknightlife.co.uk (don't get confused with the .COM one out there) Cheers Guys
On-Page Optimization | | Lukescotty0 -
Long Url but makes no sense
Hi Just joined. Crawl states that I am getting a lot of errors, looks like the spider is getting confused and looping back on itself ? Is there a way to see where the crawl was formulated (ie where from) ? It is generating urls like: http://www.wickman.net.au/wineauction/wine_auction_alert.aspx/auction/auction/auction/auction/auction/auction/Default.aspx from http://www.wickman.net.au/wineauction/wine_auction_alert.aspx
On-Page Optimization | | blinkybill0 -
Google Webmaster Tools notice of detected unnatural links
Hi Has anybody else ever recieved this message from Google in Google Webmaster Tools and what action did you take to overcome the problem and get your position back - Dear site owner or webmaster of............... We've detected that some of your site's pages may be using techniques that are outside Google's Webmaster Guidelines. Specifically, look for possibly artificial or unnatural links pointing to your site that could be intended to manipulate PageRank. Examples of unnatural linking could include buying links to pass PageRank or participating in link schemes. We encourage you to make changes to your site so that it meets our quality guidelines. Once you've made these changes, please submit your site for reconsideration in Google's search results. If you find unnatural links to your site that you are unable to control or remove, please provide the details in your reconsideration request. If you have any questions about how to resolve this issue, please see our Webmaster Help Forum for support. Sincerely, Google Search Quality Team
On-Page Optimization | | AMG1000 -
Title Tag To Long
In the initial review of my site, I have 38 warnings of title tags to long. It also says these warnings are often penalized by search engines. In Google webmaster tools, It says I have no problems with long title tags. So am I getting penalized and do I need to shorten them? I would already have shortened them but in the past, everytime I have changed my title tag I have been penalized by the search engines. I usually get my rankings back but it can take up to a month and a half to get back to where I was.
On-Page Optimization | | tkobrien0 -
How do I do a 301 Redirect in IIS 7 from http://www.freightmonster.com/index.html to http://freightmonster.com/index.html when I don't have a physical page to redirect?
I'm trying to get rid of my Rel Canonical links and use the 301 Redirect instead.
On-Page Optimization | | FreightBoy0