Google crawl rate almost zero since re-launch, organic search up 50% though!
-
We're confused as to why Google's crawl of our site has dropped hugely since our new site went live. The URLs of almost all pages changed, and were 301d to the new site. About 20% of our pages were blocked by robots.txt for the re-launch. The re-launch has been great for organic search, with hits up about 50%. Yet our new content is taking a lot longer to get indexed than before. Our KB downloaded a day according to webmaster tools are well down, as is time spent downloading a page. Any ideas as to why this is?i7hwX.png
-
Hi David,
First, my preference would always be to use 301 redirect to send pages that no longer exist to a new URL.
Using a 301 passes link juice and signals search engines that the old page should be de-indexed and the new page indexed in its place.
Barring some issue with your rel=canonical tags, which would be the first place to look, I would check other things that can send signals to the search engines about freshness:
- meta refresh tags
- update frequency reported in your sitemap
- custom crawl rate setting in Webmaster Tools
Then of course, there is an analysis of the differences between your old design and the new...have you made a significant change to your menu(s) and/or internal linking structure which has made it harder for crawlers to follow links to all pages in your site?
Hope that helps,
Sha
-
It's the same domain but we've rebuilt the website and improved the url and content structure.
We've consolidated urls (e.g. redirecting to canonical url) and published the update xml sitemap.
Pages are crawled correctly, we just can't figure out why the graph changed like that.
-
Hi David,
Just a few questions to help understand your situation:
- When you say "new site" do you mean that you have changed to a new domain?
If it is a new domain:
- Did you 301 individual pages to relevant pages on the new domain, or do you have a catch-all sending all traffic to home?
- Is the chart you uploaded from the new Webmaster Tools account (for the new domain)?
- Have you created and uploaded xml sitemap(s) for the new site?
If it is not a new domain:
- Have you created and uploaded xml sitemap(s) for the new version of the site?
- Have you checked the crawl rate setting in the Webmaster Tools account in case it has been manually altered?
These answers will give us a clearer idea of where to start.
Thanks,
Sha
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitelink search in google search for Brand name redirect me to 404, how?
Hi All, When I search my brand name in google and in google search result my site appears with sitelink and in site link there is option of search when I search any keyword in that search then that search redirect me to 404 page of my site. I found I have implemented wrong schema at category page for search action and then I fixed the bug but 5 days passed away still google showing 404 of my search action. I have not implemented schema for search action at homepage. Now please let me know what is the issue?
Technical SEO | | amu1230 -
Do you think have to re-submit my site to search engines after I made improvements?
Some time ago I started to do SEO for a one-page website and didn't get any positive result: no traffic, no filled in online booking form (yet another, multiple page website offering the same service yielded in multiple filled-in "schedule an appointment" forms). I found out my one-page website was considered to be "keyword-spamming" and converted it to a multiple page one. Its domain authority went up, but it doesn't still bring any traffic. I am thinking maybe I have to let the search engines know that it has been updated so they stop penalizing it? Do you think it might help and if yes, what exactly I should do? Will be thankful very much for any suggestion!
Technical SEO | | kirupa0 -
Why my external links are zero
What could be the possibility that my Moz crawler showing zero external link for my website http://ultimatecharter.com, i have build many links from different website and when i click them it goes to the website. My website is multi language and the landing page is http://ultimatecharter.com/en/home can this be a possible issue? regards Aqeel
Technical SEO | | Aqeel0 -
Launching Website
We are developing a new website and thinking google would not find it because of the directory we put it in (no homepage yet) and because there are no links to it. For example, we are building it in this directory example.com/wordpress/ but somehow google found it and indexed pages not ready to be indexed. What should we do to stop this until we are ready to launch? Should we just use a robots.txt file with this in it? User-agent: *
Technical SEO | | QuickLearner
Disallow: / Will this create repercussions when we officially launch?0 -
Want to Target Mobile site for Google Mobile Version and Desktop Site for Google Desktop Version
I have ecommerce site with both mobile version and desktop version. Mobile version starts with m.example.com and full version starts with www.example.com I am using same content through out both site and using 301 redirection by detecting user agent vice-versa. My both sites are accessible to crawl by any google spider. I have submitted both sites's sitemap to GWT and mobile site having mobile sitemap xml, so google can easily recognize my mobile site. Is it going to help to rank my both sites as per my expectation? I need to rank for mobile site in Google mobile and ranking for desktop site in Google desktop version. Some of pages of my mobile site are started to appearing in Google desktop version. So how I can stop them to appear in Google desktop? Your comments are highly welcome.
Technical SEO | | Hexpress0 -
How long does it take for Google to index a new site and has anyone experienced serious fluctuations in SERP within 2 weeks after launch?
Hi guys, I have recently launched my ecommerce jewellery site - www.luxuryfinejewellery.com - and noticed some serious swings in SERP over the last couple of weeks. From ranking No 2, 3 and 4 for the keyword 'luxury fine jewellery' on Google.com, the homepage periodically disappears from the Top 50 altogether. I thought it was the Sandbox, as I recently purchased the domain name, within the last 6 weeks, however the fact that it does rank on the 1st page some of the time is a mystery. Has anyone also experienced this? Could you provide some advice on what to expect until the the rankings settle. Thanks in advance, Satbir
Technical SEO | | deluxebydesign0 -
My site has vanished from google
Hi my site has vanished from google. We have been for a very long time. for example if you put in gastric band hypnotherapy then we would be first page number two and also lots of other keywords but now we have vanished from google and i do not know why or how to solve this. can anyone please help me and help me understand what i need to do to solve this please My site is http://www.clairehegarty.co.uk I am not sure if i have been banned or why i have dropped out of google
Technical SEO | | ClaireH-1848860