Does Rogerbot recognize rel="alternate" hreflang="x"?
-
Rogerbot just completed its first crawl and is reporting all kinds of duplicate content - both page content and meta title/description.
The pages it is calling duplicate are used with rel="alternate" hreflang="x", but are still being labeled as dupes.
The title and descriptions are usually exactly the same, so I am working on getting at least those translated into different languages.
I think its getting tripped up because the product page its crawling are only in English, but the chrome of the site is in the translated languages. The URLs look like so:
Original: site.com/product
Detected duplicates: site.com/fr/product, site.com/de/product, site.com/zh-hans/product
-
Hey there,
Rogerbot doesn't look for rel alts. The bot will follow meta robots, rel canonical (more used way to controlling duplicate content) and 301 redirects. Sorry about the confusion.
Best,
Nick
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Alternatives to Domain Authority
Is there another metric (or metrics) that is similar to the Moz Domain Authority that doesn't fluctuate as much? I've been reporting it on a monthly basis for about a year, but lately it seems like it's been going up and down every few weeks.
Moz Pro | | pbhatt0 -
Keyword Rankings Compare to X not working
Anyone else having trouble with the compare your keyword positions with competitors section of the analytics? The rankings for my site(s) are fine but whichever competitor I click on to compare rankings to just returns 'Not in Top 50' the competitor is ranking. I have just manually checked and they are very much listed in the engine. Is this tool broken?
Moz Pro | | GrumpyCarl0 -
Rel Canonical Question
Hi all. I think I'm a bit confused. When I check my crawl diagnostics its listing lots of warnings under the heading rel-canonical. I am not sure why, since virtually all my pages have the link rel="<a class="attribute-value">canonical</a>" tag. I use it because there are a lot of possible extensions that can appear on the pages as it is an eCommerce site. Why would seomoz list this as a warning? Thanks Ken
Moz Pro | | CandymanKen0 -
How to get rid of the message "Search Engine blocked by robots.txt"
During the Crawl Diagnostics of my website,I got a message Search Engine blocked by robots.txt under Most common errors & warnings.Please let me know the procedure by which the SEOmoz PRO Crawler can completely crawl my website?Awaiting your reply at the earliest. Regards, Prashakth Kamath
Moz Pro | | 1prashakth0 -
Does anyone know of a crawler similar to SEOmoz's RogerBot?
As you probably know SEOmoz had some hosting and server issues recently, and this came at a terrible time for me... We are in the middle of battling some duplicate content and crawl errors and need to get a fresh crawl of some sites to test things out before we are hit with the big one? Before I get a million thumbs downs- I love and will continue to use SEOmoz, just need something to get me through this week ( or until Roger is back! )!
Moz Pro | | AaronSchinke1 -
Blocking all robots except rogerbot
I'm in the process of working with a site under development and wish to run the SEOmoz crawl test before we launch it publicly. Unfortunately rogerbot is reluctant to crawl the site. I've set my robots.txt to disallow all bots besides rogerbot. Currently looks like this: User-agent: * Disallow: / User-agent: rogerbot Disallow: All pages within the site are meta tagged index,follow. Crawl report says: Search Engine blocked by robots.txt Yes Am I missing something here?
Moz Pro | | ignician0 -
SEOmoz bot and "noindex"
As a recent newbie to SEOmoz, I've been implementing some suggestions and doing a general tidy up. I removed URL's from our robots txt, and rolled out instead the noindex meta tag to pages we don't want indexed. But surprised to see issues that are now flagged from the last crawl by the moz bot from pages that have this meta tag? Does the SEOmoz bot not ignore this tag? Just want to make sure I've implemented it correctly, so the google bot does ignore it. Meta tag syntax is and is placed below the title tag. cheers Steve
Moz Pro | | sjr4x40 -
Is there a way to see all SEOMoz questions, comments, posts, responses that I've given a "thumbs up" to?
Lots of times I'll Thumbs Up a question, blog post or response not only if I think it's high quality but if I think I'd like to revisit it in the future. Essentially, I want to use a the SEO Moz Thumbs Up as a bookmark. Is there any way to see all the SEO Moz content that I have given a "Thumbs Up" to? I know I can review all the questions I've asked or answered in the "My Q&A" page. Perhaps, I'm just missing a "Thumbs Up" page somewhere on my profile?
Moz Pro | | TaitLarson2