Google Webmaster indicates robots.text access error
-
Seems that Google has not been crawling due to an access issue with our robots.txt
Late 2013 we migrated to a new host, WPEngine, so things might have changed, however this issue appears to be recent.A quick test shows I can access the file.
This is the Google Webmaster Tool message: http://www.growth trac dot com/: Googlebot can't access your site January 17, 2014 Over the last 24 hours, Googlebot encountered 62 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 8.8%
Note the above message says 'over the last 24 hours', however the date is Jan-17
This is the response from our host:
Thanks for contacting WP Engine support! I looked into the suggestions listed below and it doesn't appear that these scenarios are the cause of the errors. I looked into the server logs and I was only able to find 200 server responses on the /robots.txt. Secondly I made sure that the server wasn't over loaded. The last suggestion doesn't apply to your setup on WP Engine. We do not have any leads as to why the errors occurred. If you have any other questions or concerns, please feel free to reach out to us.Google is crawling the site-- should I be concerned? If so, is there a way to remedy this? By the way, our robots file is very lean, only a few lines, not a big deal.
Thanks!
-
Unfortunately, no. Free product = No Service
-
Cody, thank you.
And... there is no 'Google contact' for these kinds of issues, correct? (dumb question).
-
This happens from time to time on sites that I manage. I usually just check to make sure that it has been crawling since that date. If it has, then life is good.
If it hasn't, then I usually shout obscenities before trying to find the problem. The obscenities don't directly contribute to a solution, but it does seem to make me calm down a bit. Anyway, I digress.
Take a look at your webmaster tools, and see how well the site has been being crawled since that date. Is it all O.K.? If not, try to describe any changes you made in the weeks prior to that date.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Drop in Bounce Rate in Google Analytics
Hi guys, I have recently seen a large drop in bounce rate (from GA) which seems unnatural for one of our clients website. Since the start of 2018, the bounce rate was consistently between 40-60%, and then saw a random spike, and now for the past two weeks, the bounce rate is below 10%. I was wondering if anyone had any ideas if this is a problem with GA, or the site itself. Site: https://www.zoomocarcredit.com/ Any comments/feedback is greatly appreciated. Thanks in advance! Jack. SnP0Hc4
Reporting & Analytics | | ChemistryMarketing0 -
Google Analytics Campaign Tracking overriding each other
Howdy, fellow mozzers. So, here is the situation: Let's say someone comes to the website through Organic Google Search, and first thing they see is the banner on the website, on a click of which it takes you to a promotional landing page. I have this banner being tracked with GA campaigns. Now, here are my questions: If that user makes a purchase, which source / medium it will be assigned to? Organic/google or website/banner? (as far as i understand it assigns it to website/banner) If that's the case, is there way to find out afterwords where the website/banner sessions came initially from? If there is no such way, how shall i setup the tracking to be able to see wherre the initial visit to the website came from, yet also track how many people clicked on the banner and made a purchase? Thanks 🙂
Reporting & Analytics | | DmitriiK0 -
Google Search Console - > Google Search Analytic gives figure for google organic or adwords or combine of both?
Hi All, In Google Search Console -> In Search Analytics. I can see Clicks, Impressions, CTR and Position. I want to know all these 4 - Clicks, Impressions, CTR and Position gives information related to google organic only? or combine or google organic and google adwords? Thanks!
Reporting & Analytics | | wright3350 -
Google Analytics goals by source report?
Hello everybody. Is there way in Google analytics to create report on what goals have been completed per each source? Example: Lets say I have 3 goals: Subscription, Purchase, Quote. How can I get report, saying something like this: google / organic - Subscription - 5 conversions
Reporting & Analytics | | DmitriiK
Purchase - 3 conversions
Quote - 10 conversions and so on. P.S. Basically, I want the reverse of standard Google Analytics goal completions report, where you can click on goal and see which sources/mediums completions came from. I'd like to do the opposite - "click" on source/medium and see which goals have been completed. Thanks0 -
Weird Google Analytics tracking question
I have a client that has a market place site, where people list goods and sell them, think something like Etsy. Instead of developing a system to show the users page views and things like that, does it sound reasonable to let them enter a Google Analytics property on the pages they list on, then let them monitor through GA? Does anyone see any fatal flaws in this thinking?
Reporting & Analytics | | LesleyPaone0 -
Sitemap 404 error
I have generated a .xml sitemap of the site www.ihc.co.uk. The sitemap generated seems all fine, however when submitting to webmaster tools, it is returning a 404 error? anyone experienced this before. deleted and re-done the process. Tried different xml sitemap generators and even cleared cache along the way.
Reporting & Analytics | | dentaldesign0 -
Google Analytics - Cross Domain Tracking Goals
Hi guys, looking at the best way to organize this - Site 1 - Shoe Store - (Old site with analytic history - UA-12345-1) Site 2 - Clothing Store - (Brand New UA-54321-1) Right now i have 2 separate GA Accounts for the sites. But i would like to track goals from Site 2 to Site 1. For that i need to set up cross domain tracking. So i take it i would need to create another account under Site 1 - (UA-12345-2) and put that new code on Site 2? Or do the UA numbers have to be exact? example both 12345-1? In which case, i would need to set up 2 different profiles with filtering? Main concern is i want to keep separate data for both sites, espically if UA's will be the same. Trying to visualize - UA-12345-1 Profile 1Shoes/Master Profile 2 Shoes Profile 3 Clothes Or UA-12345-1 Shoes UA-12345-2 Clothes Thanks, Mike
Reporting & Analytics | | IsHot0 -
Does Google Analytics use your data against you?
I couldn't find this question answered anywhere in Q&A, so I apologize if it's a duplicate of another post. I heard, about a year ago on either Web Pro World, or Warrior Forum that Google uses your visitor data in your Google Analytic account to rank your site. Someone said that when they took out the Google code, their site went from the third to the first page within 48 hours. That was then verified over the next couple of weeks by others. Their thought was that regardless of the optimized page and incoming link, if the site wasn't getting visitors, then it would be penalized. Since Google has the data, they would be able to use it. I then started using another, paid, solution - getclicky.com. While I like clicky, there is some info Google has that clicky doesn't, everyone integrates with Google analytics - like SEOmoz, and I'm paying a monthly fee. Now that I'm a part of a community of experts, what do you think? Have you noticed Google ranking you based on your analytics data? Has anyone experienced this, or heard about it before? Because I'd like to go back to using Google analytics. Thanks!
Reporting & Analytics | | DallasBonsai0