Google cant read my robots.txt from past 10 days
-
http://awesomescreenshot.com/08d1s6aybc
hi, my robots.txt is
http://wallpaperzoo.com/robots.txt
google says it cant read and has postponed the crawl.. its been 10 days and no crawl.. please help me in solving this issue.. this is save with
-
Hi sir Google started fetching again
thank you for your help
-
What you can do is from within Google Webmaster Tools is to ask Google to fetch the URL of the sitemap and then 'submit it to the index'. That's one technique I've seen working before.
Google Webmaster Tools or Googlebot doesn't seem to crawl robots.txt files very often. Have seen it a couple of times where it could take days for them to crawl the file again after I've changed something.
-
hi sir, there was a problem on 23rd september in robots.txt which i fixed on 25th after then google bot dint come.. how much time it will take to come back again?
-
There's nothing wrong with your robots.txt file as far as I can see. It is likely that Google tried accessing the robots.txt file when your server was slow/down for a split second and that's why you are seeing the error. Are you sure that the site hasn't been crawled in 10 days? Have you checked your server log files to verify that? Regardless, once Googlebot does eventually come back, it should read the robots.txt file correctly and then you should be good to go.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google is simplifying my meta titles. What does it mean?
Hi Guys. I've noticed that google seems to be simplifying my meta titles in their SERP for some keyword searchs. Should I be worried about this? Perhaps a sign of being a little 'spammy' in my meta titles? Or is Google just aiming to bring up the most relevant looking result? Based on that specific search? Isaac.
Algorithm Updates | | isaac6630 -
Google's spell check recognize a keyword with volume
When the keyword "acls recertification" (an important keyword for our client) is typed into the Google search box, the word "recertification" is underlined in red. Note that you only need to type "acls rec" to make the red underline appear.BUT, Google does not underline the word "recertification" when it is typed into the search box alone, nor does Google underline the word "recertification" when the following keywords are searched: cpr recertification bls recertification pals recertification ^These are all closely related to the keyword "acls recertification," so this spell check behavior is very inconsistent.Why does this matter? Because no matter how close you come to typing "acls recertification," Google's autocomplete suggestions never include "acls recertification" (because of the perceived misspelling?).BUT, Google does suggest "acls recertification online" in the dropdown menu. If you select the "acls recertification online" suggestion then backspace until the word "online" is gone, the red underline disappears, and "acls recertification" becomes an autocomplete suggestion. VERY strange behavior...I have replicated this issue on various depersonalized browsers and devices, so I am confident that this is not related to my personal settings.This keyword contributes to a large portion of our client's business (they specialize in acls certification and recertification), so you can imagine how concerning this is for us. Note that until very recently (3-4 months ago), this keyword did NOT have any spell-check issues. This keyword averages 2400 searches per month according to AdWords which should be enough volume to allow Google to recognize the correct spellingI posted this issue in the Google product forums, where I was advised to submit feedback directly on the search results page via Google's "feedback" link. I have submitted this feedback to Google, but I thought I would bring this to the MOZ community as well to see if anyone has experienced a similar issue, or has any ideas as to what could be causing this issue.
Algorithm Updates | | RyanKent0 -
Google sets brand/domain name at the end of SERP titles
Hi all, I am experiencing that Google puts our domain name at the end of the titles in SERPs. So if ia have a title: "See our super cool website", Google would show "See our super cool website - Betxpert.com" in the SERPs Well. This is okay. Apart from the fact that i myself often put the brand name in the title AND the fact that Google mispells the site name. The brand is BetXpert with a upper case X...so when i get a SERP with "See our super cool website - BetXpert - Betxpert.com" I am annoyed 🙂 Any one out the know how to tell Google the EXACT brand name, such that they do not set a value the site owner does not want to have? -Rasmus
Algorithm Updates | | rasmusbang0 -
Drop in Traffic from Google, However no change in the rankings
I have seen a 20% drop in traffic from google last week (After April 29th). However when I try to analyze the rank of the keywords in the google results that send me traffic they seem to be the same. Today (6th March) Traffic has fallen further again with not much/any visible change in the rankings. Any ideas on what the reason for this could be? I have not made any changes to the website recently.
Algorithm Updates | | raghavkapur0 -
How can I check Googles Page Cache ?
Hi I use to have a handy tool in Firefox (Google Toolbar) that was very handy for checking page ranks and what date a page had been cached. For a while with the newer versions of Firefox I cannot seem to locate this useful tool, Can anybody recommend any useful tools for checking the above. Thanks Adam
Algorithm Updates | | AMG1000 -
Google penalty for one keyword?
Is it possible to get penalized by Google for a specific keyword and essentially disappear from the SERPs for that keyword but keep position for the brand (#1) and some other keywords (#4 and #7)? And how would you find out that this is what happened if there is no GWT message?
Algorithm Updates | | gfiedel0 -
Why is Google changing my title tags?
I have a few sites set up this way with their title tags: "Keyword rich phrase(s) | Company name" and Google is showing more and more of them like this in the SERPs - "Company name: Keyword rich phrase(s)" I don't see this happening to many other sites...am I hallucinating or what's going on here? Is this happening to anyone else? I don't see it necessarily affecting rankings, but for my sites with little brand recognition I want those keywords first. Bueller? Bueller?
Algorithm Updates | | NetvantageMarketing0 -
Frequency & Percentage of Content Change to get Google to Cache Every Day?
What is the frequency at which your homepage (for example) would have to update and what percentage of the page's content would need to be updated to get cached every day? What are your opinions on other factors.
Algorithm Updates | | bozzie3110