Database Crash affecting our rankings
-
Here is one for you. We had a database crash, and was down for an hour and 15 minutes, on Dec 18th. During this time our system automatically sent out a 301 redirect to our product home page. (iboats.com/boat-parts-accessories/dm/)
On Dec 24th I noticed a huge ranking drop on a lot of our keywords, and some keywords that were serving up the redirect page from our database crash, in place of its normally ranked page.
So my assumption is that Google was crawling our site during the database crash and caching the redirect pages in place of our regularly ranked pages. So when a search query is made that would normally display our ranked page, it is now displaying the redirect page from the crawled cache.
And on other keywords where we normally would rank for, Google no longer has that page cached, so it drops it from ranking.
My question: Does this assumption sound accurate? If so, I'm assuming over time that our past ranked pages will show up once again after Google recrawls those pages and saves them in their cache.
I have several sites that were affected with this:
Thanks in advance for your input
-
Glad to hear you appear to be back on the right track
-
Update:
After unblocking Google, almost all of our rankings are back up, and in some cases a little improved. I was surprised how quickly Google recrawled and ranked the pages that dropped.
I did do a Fetch and submit on all my websites and pages. I'm not sure how Fetches are attended to by Google, but it appears they are.
I have an IT guy who doesn't feel a Fetch makes any difference. He says Google will crawl when they want to and probably does not pay attention to the Fetch request.
-
I wanted to get back to report the cause of our problem with a lot of our rankings dropping. We originally thought it was due to a database crash. And that still could be affecting some things.
We moved our servers to a new location. A couple of weeks ago, we switched over to some new IP addresses. When we did so, a program started blocking Google's IP addresses because they were hitting our servers with a lot of hits, and put those IP addresses on a blocked list. So our pages on some of the websites stopped getting crawled, and thus rankings dropped. We discovered the problem when exploring our Webmaster Tools Crawl Stats and noticed that the "kilobytes downloaded per day" had flat-lined. Doing a fetch on selected pages we could see that it was not fetching the page. So after unblocking those IP addresses, the pages got crawled, our crawl stats went back up, and our rankings started going back to where they were. We still have some rankings that have not bounced back, but assume they will as new crawls re-index those pages.
A lesson learned for me was to pay attention to the crawl stats. Any drastic variations could be a sign of a problem.
Brad
iboats.com -
I'm pretty sure that GWT is a little vague in terms of times that Googlebot visited with just the crawlrate shown on a day by day basis and there's some lag between now and when it happened. CPanel will show when spiders visited in latest visitors from memory, however I think that rather than looking back it would be better to simply ensure that this doesn't happen again with the 302 rather than the 301. Maybe refreshing of your sitemap submissions in GWT would help to get some crawling going on with a ping or two to get Googlebot back soon.
-
Also, can't we use GWT to find out the exact times or at least approximate times of when Google crawled a site? Or can we not find that information so precisely? In this case, it does seem that the 301 vs. 302 is the key problem, but also the fact that the system for getting rid of the 301 didn't work out properly.
-
It seems extreme that Google would react within an hour and 15 mins but who knows nowadays. Keep an eye on your GWT account and hopefully if you have a decent crawl rate things should remedy themselves. Stephen Salstrand made a good point. 302's rather than 301's should help. It's great to have something in place like a redirection for user experience in case of database failure, however, as Stephen said, it should be made clear to Google that it's only temporary.
-
Investigating it further, we have some permanent redirects that are still hanging. I've got my IT guys working on it as we speaking.
I've got to run, but I would like to revisit this soon. Once we have determined the problem, I will post it here.
Thanks for your opinions. They have helped.
-
Your assumptions are globally right. However, I have never seen Google react that quickly to a 301 redirect or even a crash, I always tell my customers "if you have a temporary issue with 404 and 500 erros, don't worry about it too much, Google won't react unless it is a permanent issue that lasts over three or four crawls".
This is why I would also investigate a penalty that just happened to take effect shortly after the crash. To rule that out, could you please tell us if some of your major keywords are still ranking well?
I don't think you sould worry too much about that, just wait a few days and see if some pages recover.
-
Good point on the 301's. I'll look into that.
Thanks, for your input!
-
I think your assumptions are on par. However, make sure to review that the 301 has been removed. Also, if you still want that auto redirect to happen on a crash, make it a 302 and not a 301 as a 301 will suck a lot of life out of your site (since SEs see it as permanent).
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawl Attempt Errors & Homepage Not Ranking
Hi all, I have scanned the community forum thoroughly to find a solution to this issue and noticed some detailed and informed responses, but I am not sure which apply to the issue we are currently having. We are receiving a lot of 803 Crawl Attempt Errors on a weekly basis for our site www.mangofurniture.co.uk and also our homepage isn't ranking and I can't help but think that the two are linked. We have some rankings for the internal pages and have a couple of other sites that use the same template as www.mangofurniture.co.uk that are doing well with no crawl attempt errors and strong homepage rankings. There are a lot of great resources out there on the Moz forum and elsewhere but I am little unsure what applies to our problem or whether to two are linked at all. We have tried rewriting the homepage and developing the internal linking system but to no success as yet. Also, because the site is fairly new so the link profile is quite small at present. Any advice regarding this would be greatly appreciated. Many thanks in advance.
Technical SEO | | FurnitureGeek0 -
Does having no content on a mobile page have effect on the ranking.
We are about to go live with a mobile version of our webshop, mobile users will be shown an alternative version of the desktop page. At the moment we have little to no content on the mobile pages; how will this effect our ranking? (desktop and mobile page have the same URL and meta, only the "body" is different)
Technical SEO | | G.School0 -
Domain Change -- Drop in Rank
So I had a client that recently decided to change their URL to make it more "searchable"/shorter. Since doing so, I have noticed a significant drop in rankings for a number of the keyword terms we were previously ranking for. Is there a way to push the links to the new URL or is the only solution to revert back to the old URL?
Technical SEO | | maxcarnage0 -
Homepage Domain not Ranking, only subpages is?
What do you think can be causing issues that my main homepage is not showing on search results for the main search terms, only my sub-pages are showing up like the reservations page. ps. I did not get a manual penalty action email. Thanks,
Technical SEO | | EVERWORLD.ENTERTAIMENT0 -
My Alexa ranking dropped after a 301 redirect is that bad?
I had all of my non www pages redirect to the www versions. My alexa ranking dropped and keeps dropping after I did this. I'm guessing its because its tracking the non www version. Does anyone know if this is correct and should I worry?
Technical SEO | | CandleCam0 -
Sudden drop in ranking google.co.uk ranking
Anyone else had any sudden drops in rankings this week? Is there an update going on? One of my primary keywords has dropped from 6th to 49th in the google.co.uk search results. Not in webmaster tools to flag an issue. I have downloaded the links from webmaster and it does look if some content has been scraped and then linked back to us from a large number of sites that we have never sort links from. I have upload the google disavow link tool. Only one keyword appears to be effected not all of them? Any ideas? Thanks
Technical SEO | | highwayfive0 -
Devalued links or negative affect?
Hi there, I'm looking into an issue with a site that was hit after Penguin was introduced. The site lost 70% of traffic over night. The site in question seemed to have a large number of backlinks with over optimized anchor text which seems to most likely be the reason for drop in rankings. But there is also some links from blog networks here too unfortunately, so my question here really is do Google just devalue these links and discount them from consideration in their ranking algorithm or do the links still count but instead of adding positive affects in SERPs add a negative affect? My reason for this question is I'm trying to determine whether it's worth saving this website or just starting fresh with a new domain. That does bring me to another question, if I have to start fresh on a new domain is it a possibility to reuse the content from the old site? (providing I remove the URLs from Google via Webmaster tools). Any help/advice/answers here would be greatly appreciated. Thanks in advance.
Technical SEO | | jayderby0 -
Mysterious drop of website ranking in google
Usually, I don't want to bother anybody by posting silly questions on forums. But this time I really might need advice. My wife and I took over the website maintenance and e-marketing of a local air conditioning company end of March this year. Before that the applied SEO strategies were not very user friendly and a little too search engine focused (spammy keyword stuffed articles, confusing website structure, a lot of directory links). Yesterday night (May 15th) the website more or less stopped ranking. For search terms like "ac repair englewood fl" or "trane north port" and many more the website was on page 1. Here are some more details: I replaced the old website with a newer version end of April. Since some of old the url structure did not apply any longer, I did a setup of around 30 301-redirects in .htaccess. The new site seemed to rank more or less as expected. The homepage has a PakeRank of 1 (seomoz Page Authority is 31). I am working on that but good natural links just take some time. site:kobiecomplete.com still brings up all the pages Google Webmaster Tools notified me on May 12th that there was a possible outage: _"_While crawling your site, we have noticed an increase in the number of transient soft 404 errors around 2012-05-08 16:00 UTC (London, Dublin, Edinburgh). Your site may have experienced outages. These issues may have been resolved. Here are some sample pages that resulted in soft 404 errors:" The listed pages under "some sample pages" are only pages from the old website which do not exist any longer and the 301 redirect was not setup. But this should have been already any issue before, if at all.
Technical SEO | | grojoh
I added the missing 301 redirects and marked them as fixed in Google Webmaster Tools. I had a copy of the website on a testing webspace (root directory of brightsidewg.com). Even though I had robots.txt set to disallow everything and WordPress search engine privacy set to do not index / follow, the website appeared on the Google search results yesterday night instead of the original website (kobiecomplete.com). Even though brightsidewg was a few ranks worse than kobiecomplete.com was, it was still ranking.
To remove the duplicate content, I deleted everything on brightsidewg.com and requested the removal of the website in the Webmaster Tools. Now brightsidewg.com is not any longer indexed (good) but it didn't help the ranking of kobiecomplete.com. Especially the homepage and the service area pages were ranking pretty decent on Google before yesterday night. Now I can not find them at all. Only other less important pages rank on page 8+ No malware on website I did not do any big changes on the website yesterday (only really minor ones). I did not acquire any weird/paid links even though there is a new link from a PageRank 0 website which I did not setup: http://www.indo-karya.com/detail/news/2012/kombise But that alone I think would not be enough for a penalty. It almost looks like that Google applied a partial -950 filter!? I could submit the website for reconsideration to Google and tell them about the duplicate content issue with my testing webspace brightsidewg.com. What do you think about it and what shall I do? Thank you so much for any help!0