Google haveing problems accessing part of my site
-
hi my site is, www.in2town.co.uk and for a few weeks now google has had trouble accessing part of my site.
Today googlewebmaster tools tells me that google is having major problems it shows, 123 pages where access were denied.
i have spoken to my hosting company who could not find a problem, so not sure what to do now. can anyone please give me advice on what the problem may be.
any help would be great
-
That's correct. I don't know the specifics, however in this case, I would assume the reduced frequency/volume would mean less trouble dealing with the hosting provider's internal challenges.
-
i have read it time and time again and do not understand how it can help. if you reduce the crawl rate then that means you reduce the number of pages and the number of times google visits your site or am i wrong
-
Fascinating how some problems can come up like this - it's not anything I'd ever seen - curious to see whether reducing Google's crawl rate will help.
-
i have asked about this problem on the google webmaster forum
http://productforums.google.com/forum/#!mydiscussions/webmasters/BogC5OZqdyM
and people are saying that it is a crawler issue with my hosting company,
they are suggesting a number of things that could be wrong which includes
Many hosting providers need to have mechanisms to throttle/limit either the number of requests or bandwidth to their customers sites.
However in this instance, they are not allowing for search engine crawlers which typically make numerous requests in quick succession. Significantly, their server's response is technically incorrect (RFC 2616), it should either not respond, or respond with a 503 (service unavailable). Google is aware of such incorrect 403s and will eventually try to crawl them again.To mitigate the issue somewhat, you can reduce your site's crawl rate in Google's Webmaster Tools
-
This confirms the need to get an expert involved - one experienced in these types of issues - a programmer or systems administrator type expert, not necessarily an SEO expert.
-
the site has been built from a template but this problem has only started to happen in the past month and the site is a couple of years old
-
unfortunately I don't know exactly what else to suggest. Who built the site? Is it a custom site, or was it automatically built using a hosting company automated site builder?
If it's a custom site or you have a developer you've worked with to build it, speak with them. if it's a site automatically built using a hosting site building system, only they can help resolve it. If not, you'll need to get a different site built. The problem you are describing should not be happening.
-
hi, when i put in http://www.in2town.co.uk on the tool you told me about it brings up some pages that are fine and some that are 403, and the 14th one down is the url as above. that comes up with a 403 however, when i just put the url on its own, it comes up as a 200, which is strange.
Have you got any ideas on what i should say to the hosting company. i have spoken to them a number of times and all i get is, everything is fine but it is not
-
When I put that URL in, I get a "200" "OK" response. If you get a "403 forbidden, you need to speak with the site systems (server) administrator to find out if they can pinpoint the problem. Something is broken on the site at the server level to cause that. Even if it's only intermittently broken.
-
hi, thanks for this. i have used that tool and it has come up with as an example the following
http://www.in2town.co.uk/news/parenting/parents-need-to-act-against-child-obesity 403 forbidden
i am not sure why this has happened and this is the same result as i am being shown in google webmaster tools.
-
One step I would suggest is that I would suggest using a tool like Screaming Frog. It can crawl your site and report on problems or broken internal links on the site.
I would also go into Google Analytics and go into the Content/Speed/Page Speed data. I would then expand the date range to cover from before they started listing problems until the day before you run the report (the current day can lag in updating in Google Analytics).
From there, look to see if there are any specific days where Google shows the page speed average (in the timeline chart) as being unusually high. There may be days that are more problematic than others.
Finally, with that same date range in place, I would filter down the list of pages in the speed report down to show me one of the pages Google lists as being unreachable. Then do the same with some others. See if Google shows any odd page speed times on those.
I would also then go to URIValet.com and run a check on some of those pages as well to see if URIValet shows the pages reachable and returning a proper "200" "okay" header message.
This is not a guarantee fix, however it can help bring answers.
-
the site is www.in2town.co.uk www.in2town.co.uk can you try it with the link and without the link, not sure why you cannot get on the site but will be interested to find out.
this problem of google have access errors has been going on for some weeks now and i do not understand what is going on
-
Hi Claire,
I was able to view your robots.txt file but nothing else. I can't seem to get to the site. Is the URL correct?
Dana
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do you link your adaptive mobile site to Google Analytics?
With Google now saying they're putting a lot more emphasis on mobile sites, we recently got notifications from Google Webmaster Tools saying that some of our pages are not built for mobile. Some of these pages, however have an adaptive page that when you visit from a mobile phone (m.mysite.com), you're taken to instead of the desktop version. My question is, how do I let Google know that I have an adaptive site and not get penalized for poor mobile usability? I already have Google Analytics on the mobile site, I just need to somehow let Webmaster tools / Google's web crawlers know that they should be looking to my mobile site for usability, not the desktop site. Any advice is appreciated!
Technical SEO | | Ditigal_Taylor0 -
WMT "Index Status" vs Google search site:mydomain.com
Hi - I'm working for a client with a manual penalty. In their WMT account they have 2 pages indexed.If I search for "site:myclientsdomain.com" I get 175 results which is about right. I'm not sure what to make of the 2 indexed pages - any thoughts would be very appreciated. google-1.png google-2.png
Technical SEO | | JohnBolyard0 -
About how google works
Hey guyz,
Technical SEO | | atakala
I want to ask a basic question. If I search for Larry Page lets say.
I think google look for it's index for word larry and page distinct.
And mix it up. But the question ;
Can google show a result which only Larry exist on the page but any of the synonym or the stem of the Page not exist .
If it can happen how this page can be showned in larry page query. Thank you.0 -
Google using descriptions from other websites instead of site's own meta description
In the last month or so, Google has started displaying a description under links to my home page in its search results that doesn't actually come from my site. I have a meta description tag in place and for a very limited set of keywords, that description is displayed, but for the majority of results, it's displaying a description that appears on Alexa.com and a handful of other sites that seem to have copied Alexa's listing, e.g. similarsites.com. The problem is, the description from these other sites isn't particularly descriptive and mentions a service that we no longer provide. So my questions are: Why is Google doing this? Surely that's broken behaviour. How do I fix it?
Technical SEO | | antdesign0 -
Roger has detected a problem
SEOMOZ says Roger has detected a problem: We have detected that the domain www.romancebookstore.com.au does not respond to web requests. Using this domain, we will be unable to crawl your site or present accurate SERP information . What is wrong with this domain??
Technical SEO | | damientown0 -
We're no longer turning up in Google SERP for our brand search when we used to be #1 after our site update. Any ideas why?
We recently updated our website and during the push, someone mistakenly 301 redirected "www.brandx.com" to "brandx.com" instead of the otherway. Since then, our website no longer turns up for the search "brandx" on Google. We have reversed the mistake a few days ago, but we're still not turning up, and we used to rank #1 in Google SERP. Could it just be due to timing between the crawls and that our www. site didn't make it in Google's index due to this mistake? We have submitted our new sitemap to google a couple of days ago as well, as a side we're still showing up #1 in Bing's results however. And it should still show up based on SEOMoz's SERP report. Any help would help as I'm growing increasingly concerned.
Technical SEO | | JoeLin0 -
Google: site gone from SERPs, back in 1 day, then gone again?
Last December we fumbled our 404 error page with a misconfigured server and broken links on the page. Needless to say, our site dropped into an abyss - for 4 months. Yesterday we appeared again in our regular placement (actually stronger placement). Our site has been around since 09-Jan-1999 and has been a highly regarded site with good link structure and solid content for engineers. Then today we're gone again. Yesterday morning we had 125,000 pages indexed with Google which grew to 187,000 by late afternoon. Then this morning we're nowhere to be found and only 21,800 pages are now indexed. We've been working with Bruce Clay Inc through an SEO site audit doing several updates to improve through good seo practices. We haven't made any changes to the site since last Friday, April 13th (maybe the unlucky number has something to do with it....) Any ideas, insight, suggestions???? Thanks!
Technical SEO | | Prospector-Plastics1 -
How to turn WP site into Ecom site?
I have a couple of old wordpress sites that are old affiliate blogs. I currently sell products that are on amazon on these sites and sell quite a bit of volume. I have found a source and can afford the inventory to replace Amazon with my own product. So the dilema is how to turn these wordpress sites into ecommerce sites. The thing I am worried most about is that each site gets about 100-200 visitors a day for great buying keywords. I obviously don't want to lose my rankings. What are the options of turning a wordpress site into a store. I am not interested in plugins or some of the other solutions that make the store look very cheap and I would assume horribly convert. If you have inner pages ranking for keywords how does that work? Do the post pages become product pages? So to sum up I guess I am asking, what are the options that are of the higher quality that will also help me keep my rankings? Thanks
Technical SEO | | PEnterprises0