What is the appropriate Robot.txt to unblock if Google cannot get all the resources from my homepage?
-
Hello everyone. I did some research as to why my website has decreased in the Google search rankings recently. After reading this Yoast article I believe it's because the robot.txt files I have set up on my wordpress website. The following is a screen shot of the results of a fetch & render query of my webpage.Googlebot couldn't get all resources for this page. Here's a list:
My current robot.txt settings are as follows;
User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: */xmlrpc.php Disallow: */wp-*.php Disallow: */trackback/ Disallow: *?wptheme= Disallow: *?comments= Disallow: *?replytocom Disallow: */comment-page- Disallow: *?s= Disallow: */wp-content/ Allow: */wp-content/uploads/ ```What to I need to allow/disallow to allow Google spiders to properly read my website? -
Remove the following line:
Disallow: */wp-content/ And add: Allow: /wp-includes/js/
http://fonts.googleapis.com/css?family=Open+Sans:400,600,700,800%7CPT+Sans:400,400italic,700,700italic%7COswald:400,300,700&subset=latin,latin-ext
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to get gold star reviews on SERP's
Hello I have always wanted to know: How to get gold star reviews on Google in search results! Please look at the screenshot to see what I am talking about. Please can someone share with me the knowledge required to make this happen - it would be perfect for my e-commerce website! GLkOK
Algorithm Updates | | xdunningx1 -
Can Google penalize a country keyword
Hello again guys Thank you for your previous help with www.kids-academy.co.uk - we are slowly getting there! I wanted to ask something I cannot seem to find an answer to, can Google penalize you by country? By this I mean; Search term
Algorithm Updates | | LeanneSEO
Nursery franchise UAE Page 1
Nursery franchise UK Nowhere to be found! The page in question (well a section of the site) has been optimised for UK, however, as they do have a sister site in the UAE, it mentions those areas too. The pages I have been working on are now ranking reasonably well to say there is a long way to go, but for long tailed keywords NOT including anything to do with the UK. There are no naughty backlinks with the anchor text to do with the UK, the server is hosted in the UK, it is a .co.uk URL (no geotagging but I would like to know if this is of any use with this type of URL, everything says no, but it cant harm can it?) - is it possible Google due to bad practices in the past have slapped a penalty on the specific keyword area? Not something I have come across previously but I am scratching my head over here! Time for a brew break 😄 Thanks in advance guys! Leanne1 -
Does anyone have an idea of the benefits of Google Analytics Premium?
We've been having a discussion about the GA Premium service here in our office, trying to weigh up the pro's and con's... For the majority all it seems you gain access to is more support from google. We're trying to find out if that is the case or if you gain extra information, such as and insight into the search terms who must not be named. Of course i'm talking about the (Not Set) data... This section of data is ever increasing, yes i know we can access certain terms through webmasters but it was so much easier (in the good ol' days) when all the data was under one roof! Any thought opinions or even more questions would be greatly appreciated, i look forward to your responses. Anthony
Algorithm Updates | | Kal-SEO0 -
Next big Google update. Andy ideas when?
I'm interested to find out if anyone has and idea of when it will be or good guesses I would be interested to hear your thoughts. I have put a big effort into an SEO upgrade on my site and I'm interested in what impact I can expect when the next big update comes. Assuming I have fixed the issues that had been causing my traffic drop.
Algorithm Updates | | mark_baird0 -
Good results in Web Search, getting worse in Mobile Search
We are redirecting mobile visotors from http://www.1website.com to http://m.2website.com. 1website position in web SERPS is always the same. But mobile search brings less visitors now, 1website is loosing positions there. I know keywords volume is increasing. What could be the reason? (m.2website mobile version is high quality and have very good bounce, pageviews, time on site)
Algorithm Updates | | bele0 -
Google dance/over optimized/paranoid?
Hi guys, hope your all OK and thanks in advance for taking a nosey at this. OK where to start - my rankings for the last 12 months have progressively improved every week, usually of the 300 KWs i track the last few months has seen approx 70 up/70down per week, but the improvements usually outweigh the declines. This week I saw a sudden drop though - 35 improvements and 112 declines. The strange thing was though, the improvements came on the more competitive KWs, and the less competitive words I haven't done much or any back linking for dropped. Seems silly me asking this question when I run that through my head ofcouse KWs you don;t work on will drop like flies? It should be plainly obvious those words would drop off but all have been improving on there own slowly over the last 6/7 months. Now if this was a penalty (nothing showing in webmaster tools) I would have expected it to come through on my KWs I have over done the backlinking for, but these are the 1's that improved. So is it just the Google Dance? I normally see some words such as the big 1 we target DJ Equipment go from position 13 - 24 can change hourly sometimes! Could it just be quite a few have dropped all at once and will pop back up this week? Also if anyone could give us any pointers in general on where you think we should be taking our SEO it would be much appreciated. I know we have been a little lazy with our backlinking and could do with some much better/ industry related websites linking to us, and there are title tags/metas on product page that need sorting.. aside these couple of issue's? DJs Only
Algorithm Updates | | allan-chris0 -
Google Update?
We have a website that for the past several weeks has been very consistent at between 13,500 and 14,200 daily visits and this site received 15,600 last Thursday. THIS week, Monday is at 22,200, Tuesday is at 26,200, and at mid-day today (at about our traffic halfway point in the day) we're already at 14,000 today. This was a site that was bringing about 14,000 visits as of May 16th last year and dropped to 11,000 the following week. The traffic to this site this week is so far beyond statistical analysis that there must have been something that happened.
Algorithm Updates | | sourcelinemedia0 -
When did the New Google Algorithm Come into Force in the UK
On March 3rd, the hits on my UK site crashed to 40% of the previous number. I am guessing it may be down to the google algorithm change:- http://wisestartupblog.com/seo/google-algorithm-change-february-2011-losers-winners/5081 However, I am aware this happened on 25th Feb in the US. Do you know when it happened in the UK. Could it have suddenly started affecting me on March 3?
Algorithm Updates | | MattBB121
