How can I get Google to crawl my site daily?
-
I was wndering if there was a trick to getting google to crawl my website daily?
-
Great advice from everyone!!! Thanks!
Saibose - If I add a twitter, FB or blog feed to my site, and it updates daily, this will help the rate that google crawls my site? Does it matter that it is embeded content?
-
Blogs, tweets and other social mentions are always good for a fresh crawl.
-
The best way is to enable the creation of unique content daily. UGC and user participation if added to the unique content would add to the crawl ability. If you want to go a bit high tech, you can put the fresh content on a webcache to enable Google bots to visit you more frequently. ALso, make sure that your older/archived pages get some content cycled ( new comments, new tweets etc etc)
-
sitemap should show daily, but only for pages you intent to change daily. I got the idea from a Matt Cutts video, that they only honour the sitmap if they find it to be reliable. if you say daily but never update the content, then its not going to work.
-
Proper sitemap that tells the SE's to come back can help.
Write content daily. If you have a blog, ping google when you post a new piece of content. If you show Google that you write content daily, they'll come back.
I did SEO for a very successful conservative news website and Google crawled them 15-20 times a day and indexed things in sometimes seconds.
Make good use of pinging!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to Let Google Know I am a new Site Owner and to Remove or De-value all backlinks?
I am looking to buy a new domain for a brand. Problem is the domain has was registered in 1996 and has around 6k backlinks (according to ahrefs) that I need removed as the old content will have no relevance to my new site. Should I just disavow all of them? Is there anything "special" I can do to let Google know that it will be new site owner/content and to remove/discount the current links?
Technical SEO | | RichardSEO0 -
Staging site and "live" site have both been indexed by Google
While creating a site we forgot to password protect the staging site while it was being built. Now that the site has been moved to the new domain, it has come to my attention that both the staging site (site.staging.com) and the "live" site (site.com) are both being indexed. What is the best way to solve this problem? I was thinking about adding a 301 redirect from the staging site to the live site via HTACCESS. Any recommendations?
Technical SEO | | melen0 -
Can someone tell me how in the heck this site is ranking?
Just saw a huge update on the Serps for "SEO Company" and "Seo Firm " and this site has tons of big Google no's no's going to it. www.seospecialists.co it has links coming from Porn and Gambling sites. It is heavily over optimized for both keywords. I won't cry the sky is falling[SEO is broke] but I am curious to know why Google thinks they are worthy to rank that high.
Technical SEO | | MattieMac0 -
What to do about Google Crawl Error due to Faceted Navigation?
We are getting many crawl errors listed in Google webmaster tools. We use some faceted navigation with several variables. Google sees these as "500" response code. It looks like Google is truncating the url. Can we tell Google not to crawl these search results using part of the url ("sort=" for example)? Is there a better way to solve this?
Technical SEO | | EugeneF0 -
Htaccess redirects how many can i have and does it slow site down
I have had to redesign my site this year www.in2town.co.uk because my hosting company made a huge mistake while trying to update the joomla on my site which resulted in me losing thousands of pages and links. What i would like to know is, i have put some of the old urls in my htaccess file but i would like to know how many old urls i can have in my htaccess file as i am unsure how to use it properly. my idea was, to have some of the lost url links to my site and put them in my htaccess file and have them pointing to similar pages. not sure if this is a good idea or not. i think i have lost a few hundred good links but i would like to know if these urls in the htaccess file would slow down my joomla site any advice would be great.
Technical SEO | | ClaireH-1848860 -
Mobile site rank on Google S.E. instead of desktop site.
Hello, all SEOers~ Today, I would like to hear your opinion regarding on Mobile site and duplicate contents issue. I have a mobile version of our website that is hosted on a subdomain (m instead www). Site is targeting UK and Its essentially the same content, formatted differently. So every URL on www exists also at the "m" subdomain and is identical content. (there are some different contents, yet I could say about 90% or more contents are same) Recently I've noticed that search results are showing links to our mobile site instead of the desktop site. (Google UK) I have a sitemap.xml for both sites, the mobile sitemap defined as follows: I didn't block googlebot from mobile site and also didn't block googlebot-mobile from desktop site. I read and watched Google webmaster tool forum and related video from Matt Cutts. I found many opinion that there is possibility which cause duplicate contents issue and I should do one of followings. 1. Block googlebot from mobile site. 2. Use canonical Tag on mobile site which points to desktop site. 3. Create and develop different contents (needless to say...) Do you think duplicate contents issue caused my mobile site rank on S.E. instead of my desktop site? also Do you think those method will help to show my desktop site on S.E.? I was wondering that I have multi-country sites which is same site format as I mentioned above. However, my other country sites are totally doing fine on Google. Only difference that I found is my other country sites have different Title & Meta Tag comparing to desktop site, but my UK mobile site has same Title & Meta Tag comparing to desktop. Do you think this also has something to do with current problem? Please people~! Feel free to make some comments and share your opinion. Thanks for reading my long long explanation.
Technical SEO | | Artience0 -
Removing a site from Google's index
We have a site we'd like to have pulled from Google's index. Back in late June, we disallowed robot access to the site through the robots.txt file and added a robots meta tag with "no index,no follow" commands. The expectation was that Google would eventually crawl the site and remove it from the index in response to those tags. The problem is that Google hasn't come back to crawl the site since late May. Is there a way to speed up this process and communicate to Google that we want the entire site out of the index, or do we just have to wait until it's eventually crawled again?
Technical SEO | | issuebasedmedia0 -
"Site Suspended" in Google Adwords + Lost all rankings in Google => is this related?
Can anyone share thoughts on this: Does the S recently (mid april) we revamped our website (same content, new layout, strong brand), but a few days later our google rep contacted us to tell that she got a "red flag" for one of our SEA campaigns (we broke the bridge page policy, not on purpose to be clear), they were completely correct on this matter. We even got some extra time to correct this, normal policy is only 10 days. But, we were a little slow, so all our Adwords Campaigns are suspended and we get the message "Site suspended". We are working to have this fixed, our Google rep even granted some more time to fix this. Now, almost simultaneously, same time frame, all our new pages, that were already ranking well tx to proper 301 rules, suddenly fell out of the google SERPS, nothing to be found anymore up till now. Our website is live since 1996, no issues, up till now. There seems to be a strong correlation to what happened in our SEA and what happened in our SEO can anyone share some info?
Technical SEO | | TruvoDirectories0