Meta Robots "Index, Follow"
-
In my MozBar under "General Attributes" it says "index, follow" next to Meta Roberts for one of our client's websites. I've never seen "index, follow" before. I've seen it say "not found." What does index, follow mean and is that a bad thing? I know the reason should be obvious but this site has had a lot of problems and I'm wondering if this is related.
-
Hello!
Index/Follow means that you are allowing search engines to index your pages and follow links through those pages. If you didn't have this tag on the page, search engines would do it anyway. Basically, this meta tag isn't necessarily a bad thing and the search engines would be doing it even if the tag wasn't there.
You can use meta robots to do good things for your site, such as stop the crawling of blog archive pages to limit duplicate content. There's tons of resources here on MOZ about the robots file and I would suggest looking into it. This is a good one: http://moz.com/learn/seo/robotstxt
What type of problems is your site having?
-
Thank you! That was very helpful!
-
What you're seeing is the page's meta robots tag, which instructs the search engine bots to (in this case) index a specific page and follow links on the page. The opposite, "noindex,nofollow" instructs the search bots to not include this page in its index or follow links on the page. You can also use "noindex, follow" so the search engine keeps the page out of its index, but still follows the links on the page. I personally have never come across an example of "index, nofollow" but I imagine that it's possible - probably more of an outdated practice though related to hoarding links.
If you've never seen this attribute before, it may be because the page doesn't include the tag (in which case the bots, by default, assume "index, follow"). And as for whether or not having it is a bad thing - probably not, unless it's a page you really don't want visible in search results.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz claims we have meta noindex but we don't
Hi, I'm encountering an issue where moz scan says we have meta noindex, but I have confirmed across several of our pages that this simply isn't true. I have confirmation that the below tag is present: name="robots" content="index, follow" /> I also verified our https header through https://www.webconfs.com/http-header-check.php and see nothing indicating that we are sending any no index headers. Why would the crawler report this when it doesn't seem to be the case? Let me know if I need to provide more information.
Moz Bar | | charper_floqast0 -
What does "Optimal Use of Keywords in Header Tags" really mean?
I get this explanation on the grader: Although using targeted keywords in H1 tags on your page does not directly correlate to high rankings, it does appear to provide some slight value. It's also considered a best practice for accessibility and helps potential visitors determine your page's content, so we recommend it. Over-using keywords, however, can be perceived as keyword stuffing (a form of search engine spam) and can negatively impact rankings, so use keywords in H1 tags two or fewer times. To adhere to best practices in Google News and Bing News, headlines should contain the relevant keyword target and be treated with the same importance as title tags. But I have the keyword in an H1 tag, only once. So why am getting this message on the grader? The URL is https://www.armsecuritysystems.com/business-security-systems/ The keyword is "business security systems" Any insight would be greatly appreciated.
Moz Bar | | bizmarquee1 -
Hald the keywords in my keyword list say "Gethering Metrics". How do I get that to complete?
I'vwe waited about three hours, with no change. Why does this happen — is it something I did? Will those metrics ever be gathered? Or do I need to re-do buildingthe list all over again? And how can I be sure it will be any dirfferent?
Moz Bar | | btreloar0 -
How does the grader tool treat keyword "stuffing" in ecommerce
We recently started using Moz on our ecommmerce site because I'm concerned that our SEO company doesn't really know what they are doing and I want to see what I can do on my own with the little bit of knowledge I have. It's helping in a number of ways but here's a big question mark: The Grader Tool keeps telling us that our product category pages have too many keywords on them. We are only using them in the content once or twice, but the sub-category buttons on the page show the category + sub repeated. Could this be what's causing it? Does Google distinguish this for ecommerce? We've taken a huge hit in rankings for key phrases and keywords over the past six months and I'm wondering if this is part of it?
Moz Bar | | Creative-Web-Stores0 -
Will robots.txt override a Firewall for Rogerbot?
Hey everybody. Our server guy, who is sorta difficult, has put these ridiculous security measures in place which lock people out of our website all the time. Basically if I ping the website too many times I get locked out, and that's just on my own, doing general research. Regardless, all of our audits are coming back with 5xx errors and I asked if we could add rogerbot to the robots.txt. He seems to be resistant to the idea and just wants to adjust the settings to his firewall... Does anybody know if putting that in the Robots.txt will override his firewall/ping defense he has put in place? I personally think what he has done is WAY too overkill, but that is besides the point. Thanks everybody.
Moz Bar | | HashtagHustler0 -
Meta Refresh
Hello, Can anybody tell me what is "meta refresh"?? I found this word in SEOmoz crawl test report sent to me by moz. For every URL there is meta refresh tab saying "No". Is this good thing for bad for website?? your knowledge on this subject would help me tremendously. Thanks
Moz Bar | | acelerar0 -
"Avoid Keyword Self-Cannibalization" - can't find the problem
Hi, I understand what this means (or at least I think I do!), but I can't find where the problem lies. The keyword is "fire warden training" and the url is http://www.tutis-fire.co.uk/fire-warden-training-courses/ If anyone could lend a helping hand, I'd appreciate it.
Moz Bar | | Gordon_Hall0 -
Since the revised website was launched, I can't find the "Crawl Test" function showing Titles and Descriptions of other websites. Anyone know where that link is located?
MOZ can "crawl" any website and show information like Title, Description, etc.....Can't find that link.
Moz Bar | | bpedrazas0