Google cant read my robots.txt from past 10 days
-
http://awesomescreenshot.com/08d1s6aybc
hi, my robots.txt is
http://wallpaperzoo.com/robots.txt
google says it cant read and has postponed the crawl.. its been 10 days and no crawl.. please help me in solving this issue.. this is save with
-
Hi sir Google started fetching again
thank you for your help
-
What you can do is from within Google Webmaster Tools is to ask Google to fetch the URL of the sitemap and then 'submit it to the index'. That's one technique I've seen working before.
Google Webmaster Tools or Googlebot doesn't seem to crawl robots.txt files very often. Have seen it a couple of times where it could take days for them to crawl the file again after I've changed something.
-
hi sir, there was a problem on 23rd september in robots.txt which i fixed on 25th after then google bot dint come.. how much time it will take to come back again?
-
There's nothing wrong with your robots.txt file as far as I can see. It is likely that Google tried accessing the robots.txt file when your server was slow/down for a split second and that's why you are seeing the error. Are you sure that the site hasn't been crawled in 10 days? Have you checked your server log files to verify that? Regardless, once Googlebot does eventually come back, it should read the robots.txt file correctly and then you should be good to go.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does Google considers the direct traffic on the pages with rel canonical tags?
Hi community, Let's say there is a duplicate page (A) pointing to original page (B) using rel canonical tag. Pagerank will be passed from Page A to B as the content is very similar and Google honours it hopefully. I wonder how Google treats the direct traffic on the duplicate Page A. We know that direct traffic is also an important ranking factor (correct me if I'm wrong). If the direct traffic is high on the duplicate page A, then how Google considers it? Will there be any score given to original page B? Thanks
Algorithm Updates | | vtmoz0 -
Where has Google found the £1.00 value for the penny black? Is it Google moving beyond the mark-ups too?
Hi guys, I am curious, so am wondering something about the Penny Black SERPs.
Algorithm Updates | | madcow78
Apparently Google shows a value of £1.00 Penny Black SERP From where does it come from? It's not the value Penny Black Value SERP The Wikipedia page hasn't any mark-up about it, actually it has the Price value mark-up of 1 penny Penny Black Wiki Markup Among the rare stamps, also the Inverted Jenny shows a value Inverted Jenny SERP But it's clearly taken from USPS and it's the cost of a new version of this rare stamp USPS Inverted Jenny Indeed, the mark-up matches that value USPS Inverted Jenny Mark-up I've been looking on-line for a new version of the Penny Black, but couldn't find anything.
The only small piece of information that I've found to correlate one pound with the Penny Black is on the Wikipedia page, but the point is: is Google able to strip those information from that piece? It's not a mark-up, it's not a number and mostly it's not a simple sentence like "The penny black cost was of £1.00" It reads "One full sheet cost 240 pennies or one pound sterling". Penny Black Wikipedia particular Is it Google moving beyond the mark-ups too? Thanks, Pierpaolo 9Cm3MOs.jpg f7XYNtF.jpg 5PpwapB.jpg hYUJswI.jpg 7kbIC4Q.jpg jnu1Gbe.jpg Wzltg0t.jpg2 -
New Zealand rankings - Is Google taking city into account now?
I've just noticed over the last 24 hours that Google New Zealand seems to be taking city into account with the search results - when I search from Christchurch - I get CHCH companies and Auckland results when searching from Auckland. I would like to rank my client nationally - any tips? Has anyone else noticed this? I need to be the top of Google for my keyword throughout the country. Any articles that could help? Kind regards, David
Algorithm Updates | | David_Buckingham0 -
How Does Google Treat External Links to URLs with # Anchors?
Here are two URLs to explain this example: **Original URL: **example.com/1/ **URL that points to anchor within the webpage above: **example.com/1/#anchor Does Google treat these two URLs as separate entities or the same? For example, does an external link to the anchor URL pass full PageRank value to the original URL? How does Google handle this? Is there anything negative about this situation? Are there any risks associated with links to the anchor URL? Finally, is it more valuable for an external link to point to the URL without an anchor?
Algorithm Updates | | SAMarketing0 -
Google Search CTR % By Position
Hello I am looking for an updated report regarding the CTR % by position for Google search results. I have the compete.com report which Gives the 1st organic position a 53% CTR but I have not be able to duplicate that number with any other report or research. I am just trying to validate this report before I suggest any recommendations to my company regarding our search efforts. Thank you Ben
Algorithm Updates | | bhalverson30 -
Decent rankings in Google, nothing in Bing and Yahoo
Hi there, I'm in the process of SEOing a site in a very competitive sector, the short term loans market. The URL for the site is http://www.piggy-bank.co.uk. I've managed to get a fair bit of success in Google for some very competitive keywords like short term loans, short term lender etc but in Bing and Yahoo I'm having no luck at all, with only 2 visits in the past month and no decent rankings!! I think I'm doing everything right, with regular new content on the site, decent technical SEO, semantic site structure, regular site map upload, a 10 year old domain, holistic link building through guest blogging etc, but still no luck at all. Looking at the webmaster tools in Bing, 95% of the URLs are indexed, but I'm getting such a low impression count, and obviously, an even lower click through. Am I missing something really obvious? Does anyone have any suggestions to improve my Bing and Yahoo rankings? I've worked on 100s of other sites and Yahoo and Bing tend to be the easy win to make the client happy 😉 Thanks in advance for your help. Dan
Algorithm Updates | | djslimited1 -
Has Google problems in indexing pages that use <base href=""> the last days?
Since a couple of days I have the problem, that Google Webmaster tools are showing a lot more 404 Errors than normal. If I go thru the list I find very strange URLs that look like two paths put together. For example: http://www.domain.de/languages/languageschools/havanna/languages/languageschools/london/london.htm If I check on which page Google found that path it is showing me the following URL: http://www.domain.de/languages/languageschools/havanna/spanishcourse.htm If I check the source code of the Page for the Link leading to the London Page it looks like the following: [...](languages/languageschools/london/london.htm) So to me it looks like Google is ignoring the <base href="..."> and putting the path together as following: Part 1) http://www.domain.de/laguages/languageschools/havanna/ instead of base href Part 2) languages/languageschools/london/london.htm Result is the wrong path! http://www.domain.de/languages/languageschools/havanna/languages/languageschools/london/london.htm I know finding a solution is not difficult, I can use absolute paths instead of relative ones. But: - Does anyone make the same experience? - Do you know other reasons which could cause such a problem? P.s.: I am quite sure that the CMS (Typo3) is not generating these paths randomly. I would like to be sure before we change the CMS's Settings to absolute paths!
Algorithm Updates | | SimCaffe0 -
How much does Google take Social into account in serps
I have been reading and trying to learn more about how google takes social media into account when ranking sites, but I cant seam to find a definitive answer to this, does it make a big difference or does it not really matter?
Algorithm Updates | | MiracleCreative0