Wordpress Robots.txt Sitemap submission?
-
Alright, my question comes directly from this article by SEOmoz
http://www.seomoz.org/learn-seo/r...
Yes, I have submitted the sitemap to google, bing's webmaster tools and
and I want to add the location of our site's sitemaps and does it mean that I erase everything in the robots.txt right now and replace it with?
<code>User-agent: * Disallow: Sitemap: http://www.example.com/none-standard-location/sitemap.xml</code>
<code>???</code>
because Wordpress comes with some default disallows like wp-admin, trackback, plugins. I have also read this, but was wondering if this is the correct way to add sitemap on Wordpress Robots.txt. [http://www.seomoz.org/q/removing-...](http://www.seomoz.org/q/removing-robots-txt-on-wordpress-site-problem) I am using Multisite with Yoast plugin so I have more than one sitemap.xml to submit Do I erase everything in Robots.txt and replace it with how SEOmoz recommended? hmm that sounds not right. like
<code>
<code>
<code>User-agent: *
Disallow: </code>Sitemap: http://www.example.com/sitemap_index.xml</code>
<code>``` Sitemap: http://www.example.com/sub/sitemap_index.xml ```</code>
<code>?????????</code> ```</code>
-
Hi!
Definitely don't delete what's in the robots.txt file now. Simply add the sitemap lines at the top of the file, one per line - like this example from the WordPress documentation
Leave everything is intact below it that is already in your robots.txt file. You probably don't have as much "stuff" as this example, so just add your sitemaps at the top and you should be all set.
-Dan
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Recipe Wordpress Plugin - Structured Data?
We have been using this plugin for years: https://wordpress.org/plugins/ziplist-recipe-plugin/ and it appears that it's causing us serious problems in Google. So we're looking for either a new plugin or possible directions on if we should custom build an option into our site. Ultimately we're looking for the best setup for the Recipes on our site to obtain a Featured Snippet. Thank you!
Technical SEO | | FabulesslyFrugal0 -
Block Domain in robots.txt
Hi. We had some URLs that were indexed in Google from a www1-subdomain. We have now disabled the URLs (returning a 404 - for other reasons we cannot do a redirect from www1 to www) and blocked via robots.txt. But the amount of indexed pages keeps increasing (for 2 weeks now). Unfortunately, I cannot install Webmaster Tools for this subdomain to tell Google to back off... Any ideas why this could be and whether it's normal? I can send you more domain infos by personal message if you want to have a look at it.
Technical SEO | | zeepartner0 -
Robots.txt checker
Google seems to have discontinued their robots.txt checker. Is there another tool that I can use to check my text instead? Thanks!
Technical SEO | | theLotter0 -
Robots.txt and joomla
Hello, I use joomla for my website and automatically all those files are blocked is that good or bad, so I remove anything and if so why ? User-agent: *
Technical SEO | | seoanalytics
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/ I also added to my robots.txt files my email address ( is that useful, I am afraid google passes PR to the email address )
and a javascript: void (0) because I have tabs on my webpage ( is that useful )
as well as a .pdf ( is it also useful ) any comments ? does anything need to be changed or is it ok ? Thank you,0 -
Using Robots.txt
I want to Block or prevent pages being accessed or indexed by googlebot. Please tell me if googlebot will NOT Access any URL that begins with my domain name, followed by a question mark,followed by any string by using Robots.txt below. Sample URL http://mydomain.com/?example User-agent: Googlebot Disallow: /?
Technical SEO | | semer0 -
Duplicate content - wordpress image attachement
I have run my seomoz campaign through my wordpress site and found duplicate content. However, all of this duplicate content was either my logo or images and no content with addresses like /?attachement_id=4 for example . How should I resolve this? thank you.
Technical SEO | | htmanage0 -
How to write 301 redirects in WordPress
I've successfully migrated new site to new domain (www.cmsearchmarketing.com) But I cannot get 301 redirects for pages and blog posts to redirect from the old domain (www.creativemindsearchmarketing.com). And it's my understanding I need to do a 301 for each page to maintain SEO. Here's what I've tried: RewriteCond %{QUERY_STRING} ^p=975$RewriteRule ^index.php$ http://www.cmsearchmarketing.com/top-5-questions-to-ask-an-seo-firm-before-signing-up/? [R=301,L] BEGIN WordPress<ifmodule mod_rewrite.c="">RewriteEngine OnRewriteBase /RewriteCond %{REQUEST_FILENAME} !-fRewriteCond %{REQUEST_FILENAME} !-dRewriteRule . /index.php [L]</ifmodule># END WordPress #AND ALSO# Use PHP5 Single php.ini as defaultAddHandler application/x-httpd-php5s .php BEGIN WordPress<ifmodule mod_rewrite.c="">RewriteEngine OnRewriteBase /RewriteCond %{REQUEST_FILENAME} !-fRewriteCond %{REQUEST_FILENAME} !-dRewriteRule . /index.php [L]</ifmodule># END WordPress redirect 301 /top-5-questions-to-ask-an-seo-firm-before-signing-up http://www.cmsearchmarketing.com/top-5-questions-to-ask-an-seo-firm-before-signing-up/ Any suggestions would be appreciated. _Cindy P.S. Maybe some other issues are in the way: --Old site is WP-Remix theme no longer supported, and latest WP version is 2.9.1 -- Old domain (www.creativemindsearchmarketing.com) is the primary account on BlueHost …and the new domain (www.cmsearchmarketing.com) is an addon, so the new domain's directory is within root of old domain. -- in root domain of old site there are other "handler files" that also have base file rewrites, if this is an issue: name of this file in root directory is:
Technical SEO | | CeCeBar
.htaccess.addHandlerBak -FrontPage- <limit get="" post="">order deny,allowdeny from allallow from all</limit><limit put="" delete="">order deny,allowdeny from all</limit>AuthUserFile /home/creatjo7/public_html/_vti_pvt/service.pwdAuthGroupFile /home/creatjo7/public_html/_vti_pvt/service.grp# BEGIN WordPress<ifmodule mod_rewrite.c="">RewriteEngine OnRewriteBase /RewriteCond %{REQUEST_FILENAME} !-fRewriteCond %{REQUEST_FILENAME} !-dRewriteRule . /index.php [L]</ifmodule> END WordPressAuthName creativemindsearchmarketing.comIndexIgnore .htaccess /.?? *~ *# /HEADER /README /_vti0 -
Best Dynamic Sitemap Generator
Hello Mozers, Could you please share the best Dynamic Sitemap Generator you are using. I have found this place: http://www.seotools.kreationstudio.com/xml-sitemap-generator/free_dynamic_xml_sitemap_generator.php Thanks in advanced for your help.
Technical SEO | | SEOPractices0