What is the appropriate Robot.txt to unblock if Google cannot get all the resources from my homepage?
-
Hello everyone. I did some research as to why my website has decreased in the Google search rankings recently. After reading this Yoast article I believe it's because the robot.txt files I have set up on my wordpress website. The following is a screen shot of the results of a fetch & render query of my webpage.Googlebot couldn't get all resources for this page. Here's a list:
My current robot.txt settings are as follows;
User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: */xmlrpc.php Disallow: */wp-*.php Disallow: */trackback/ Disallow: *?wptheme= Disallow: *?comments= Disallow: *?replytocom Disallow: */comment-page- Disallow: *?s= Disallow: */wp-content/ Allow: */wp-content/uploads/ ```What to I need to allow/disallow to allow Google spiders to properly read my website? -
Remove the following line:
Disallow: */wp-content/ And add: Allow: /wp-includes/js/
http://fonts.googleapis.com/css?family=Open+Sans:400,600,700,800%7CPT+Sans:400,400italic,700,700italic%7COswald:400,300,700&subset=latin,latin-ext
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Number or percentage of new visitors impact Google rankings?
Hi all, Does the number/percentage of new visitors (from different IPs and countries) impact the Google rankings? If there are more number of new visitors, will Google favours the website in rankings considering the fact that new visitors are better than returning/same visitors? Thanks
Algorithm Updates | | vtmoz0 -
What happens when non-relevant topic is getting more visitors?
Hi all, So we have a sub-domain which has user generated content like forums. Mostly the content is all about our product. Few times some spammy threads get posted and we delete them regularly. I have noticed that a non-relevant thread has been posted which about a movie. But this page got hundreds of clicks. I just wonder will this hurts being off topic and movie torrent thread or helps being receiving hundreds of visitors? Thanks
Algorithm Updates | | vtmoz0 -
Why are Google Webmaster Tools' Google rankings different to actual Google rankings?
Dear Moz, We have noticed that according to Google Webmaster Tools one of our client sites is ranking very prominently for some of the major key phrases that we are trying to rank them for. However, when we perform a Google search for these queries, our client's content is nowhere to be seen, not even on the 5th page (we logged out of the Google account before performing the test). A long-term manual spam action on our client's site was recently lifted by Google - is it possible that Google Webmaster Tools is providing data about our client's estimated Google rankings, without taking into consideration the penalty of the manual spam action which was taken? Thanks
Algorithm Updates | | BoomDialogue690 -
7 Pack Google Serps?
What is the best way to get into the 7 pack of google serps? I have a site that ranked well before this changed but not was pushed back to page 2. I have Unique content and I currently have provided my info to all the standard local sites, like Yelp, Manta, Local.com and others. I already have a Google Local page and I also have links from local sites. What else can be done?
Algorithm Updates | | bronxpad0 -
Data on Google Vs Bing, et al and changes to sites.
I am curious to know if anyone has any data that correlates site/page changes like content or Title Tag, H1, etc. and subsequent movement in rankings on Google and Bing and Yahoo? The equation is for example: ABCSite.com/home-page/ makes a change to the H1 and H2 and one paragraph of content is changed. Over next 6 to 12 weeks changes in page rank for the 3 engines is tracked to see where it started and where it "stopped." Obviously, there are more factors than individual algorithms in play here. An example of that would be that a significant number of sites will be indexed in Google by a dev and not in the others. We see this regularly. So, at least from a timing standpoint, different sites are entering/leaving the fray at different rates. We are going to begin to track this but I would love to see any data already around or speak with anyone involved in such a study about what they found. Thanks
Algorithm Updates | | RobertFisher0 -
Google's reaction to site updates
Hi, Is it safe to assume as soon as Google indexes updates I've made to my site that any ranking changes the updates effected will happen at that same time, or is there ever a lag time before these changes ( if any ) take effect?
Algorithm Updates | | minutiae0 -
What do you think of Google SERP encryption?
Really interesting post by Search Engine Land about this "issue" for tracking conversion, especially for long tail keyword research. I suppose this change will be also applied on all google search pages (.ca, .fr etc.). I Really don't think Webmaster tools is a serious compensation in Analytics for this.
Algorithm Updates | | Olivier_Lambert0 -
Google Update on the 6th July
Hi Mozzers, Has anyone noticed a Google update on the 6th July? A price comparison site I optimise has fallen off the SERPs for most generic terms, however still getting traffic for longer tail phrases. Cheers Aran
Algorithm Updates | | Entrusteddev0