Why are these m. results showing as blocked?

[nicole.healthline]

If you go to http://bit.ly/173gdWK, you'll see that m. results are showing as blocked by robots.txt, but we don't have anything in our robots.txt file that specifies to block m. results. Any ideas why these URLs show as blocked?

[FedeEinhorn]

Yeah, I was testing exactly the same when you posted the response. I even tried crawling a googlebot-mobile and still I get the 301 redirect. Which, for everything that I am seeing it is correct, as no matter what browser I use (desktop, mobile, spider) I always get a 301 to the www. version.

@michelleh, are you sure there's a mobile version not being redirected to the www. one?

[john4math]

(Using example.com instead of your domain in case you want anonymity later)

If you try to go to any of these m.example.com URLs on your desktop computer, you're redirected on the server to a www.example.com URL.  I'm guessing Googlebot and Googlebot-Mobile cannot access m. pages (unless you're sniffing out Googlebot-Mobile specifically to serve it m.example.com pages).  If you're looking at screen resolution for these redirects, you might not be catching Googlebot-Mobile, as I don't think Googlebot-Mobile gives a screen resolution in its user agent.  I believe you want Googlebot indexing your www. content, and Googlebot-Mobile indexing your m. content, so you'll need to sniff out Googlebot-Mobile's user agent (see here), and redirect it to m. content.

Also of note is that I think these should be 302 temporary redirects, and not 301 permanent redirects between your www. and m. versions, as they're not really permanent, just getting a given user to the right version of the site.  Also, you don't let me switch from the mobile version to the desktop version, which drives me bananas!  Let users choose after the initial redirect.  If you allow people to switch, but maintain 301 redirects, the browsers may cache some of the redirects which will lead to weird behavior if people hit a page that redirected before.

You don't have a robots.txt file at m.example.com/robots.txt, as that redirects to www.example.com/robots.txt even on my phone.  I don't think this is the root of the problem, but once you figure things out, you can set up a robots.txt file on your m. subdomain.