How well does Google's "Locale-aware crawling by Googlebot" work?
-
Hello,
In January of this year Google introduced "Locale-aware crawling by Googlebot."
https://support.google.com/webmasters/answer/6144055?hl=e
Google uses different crawl settings for sites that cannot have separate URLs for each locale. ......... This is basically for sites that dynamically render contend on the same URL depending on the locale and language (IP) of the visitor. If e.g. a visitor was coming from France, the targeted page would load in french. If a visitor was coming from the US the same page would load in English on the same URL.
Does anyone have any experience with this setup and how well it works?
How well do the different versions of a page get indexed, and how well do those pages rank?
In the example above, does the french content get indexed correctly?
Many thanks!
-
Many thanks! I was looking for examples of issues with this setup.
Some stakeholders find the dynamic solution appealing (as it is much easier to set up for us), thereby forgoing the opportunity to set up the site correctly for international indexing and ranking. I would like to be able to illustrate the potential shortcomings.
Many thanks again!
-
No personal experience with it - but seen already some cases on the Q&A where it generated problems. In one specific case all traffic outside Ireland was shown a default language page - inside Ireland the full content was available. None of the content was indexed by Google until the ip blocking was removed.
The note on the page you mention already gives a hint:
IMPORTANT: We continue to support and recommend using separate locale URL configurations and annotating them with rel=alternate hreflang annotations.
If your website has pages that return different content based on the perceived country or preferred language of the visitor (i.e., you have locale-adaptive pages), Google might not crawl, index, or rank all of your locale-adaptive content.
You can try it but you run a huge risk that your content will not be indexed or only partially indexed. Better to use the hreflang solution - as propose by Google.
Dirk
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Advise on the right way to block country specific users but not block Googlebot - and not be seen to be cloaking. Help please!
Hi, I am working on the SEO of an online gaming platform - a platform that can only be accessed by people in certain countries, where the games and content are legally allowed.
International SEO | | MarkCanning
Example: The games are not allowed in the USA, but they are allowed in Canada. Present Situation:
Presently when a user from the USA visits the site they get directed to a restricted location page with the following message: RESTRICTED LOCATION
Due to licensing restrictions, we can't currently offer our services in your location. We're working hard to expand our reach, so stay tuned for updates! Because USA visitors are blocked Google which primarily (but not always) crawls from the USA is also blocked, so the company webpages are not being crawled and indexed. Objective / What we want to achieve: The website will have multiple region and language locations. Some of these will exist as standalone websites and others will exist as folders on the domain. Examples below:
domain.com/en-ca [English Canada]
domain.com/fr-ca [french Canada]
domain.com/es-mx [spanish mexico]
domain.com/pt-br [portugese brazil]
domain.co.in/hi [hindi India] If a user from USA or another restricted location tries to access our site they should not have access but should get a restricted access message.
However we still want google to be able to access, crawl and index our pages. Can i suggest how do we do this without getting done for cloaking etc? Would this approach be ok? (please see below) We continue to work as the present situation is presently doing, showing visitors from the USA a restricted message.
However rather than redirecting these visitors to a restricted location page, we just black out the page and show them a floating message as if it were a model window.
While Googlebot would be allowed to visit and crawl the website. I have also read that it would be good to put paywall schema on each webpage to let Google know that we are not cloaking and its a restricted paid page. All public pages are accessible but only if the visitor is from a location that is not restricted Any feedback and direction that can be given would be greatly appreciated as i am new to this angle of SEO. Sincere thanks,0 -
Which would Google prefer In my case? Country Specific TLD or SubFolders?
I'm looking for some expert advice regarding multilingual SEO domain selection. I have a basic question that I'd love some help clarifying. I'd love to know what you do if you were in my position..From the research I've done so far, although there are other options, the two best suites ways of us separating 2 languages within our site is: **Country specific TLD's. (.com & .fr) ** SubDomain Folders (.com & .com/fr) Would google prefer the power of the country specific domains & the cleanliness of the separation (Option 1)? Or would it value more the link authority sent to one main domain with languages separated by subdomains (Option 2)? **Question background details:**I am developing a website in French & English. The main target markets language at present is French.In the future however I'm sure equal if not more website users will use the English language.1) Languages on two separate TLD's (Top level Domains) for each country (.fr & .com). We already own both domains We use WPML on wordpress so it's easy to update both languages. Languages in sub folder .com (en) .com/fr (fr) Through link building, all 'link juice' will be directed to .com (across french & English). We want all our customers to land on .com/fr if they are in French speaking country.
International SEO | | FullSteamBusiness2 -
Trying To Use Parent Company's Content In Another Country
Hi, I'm hoping someone can help me out here but this is what I am dealing with: Say John Smith Companies sells Widgets across the United States. They have also formed a company called "Widgets of Canada" in an effort to sell their Blue Widgets only in Canada and I am in responsible for their website. Recently, John Smith Companies completely redesigned their website and it now has a really slick look and is loaded with great widgets content. I would like to take their site and re-purpose it for use in Canada. However, I am concerned about duplicate content. I would be converting all the widget specifications from imperial to metric units, changing the title and description elements and also using a much different folders/ paths. Is this enough to avoid any issues with similar page content? Is there anything I can do with hreflang? Thanks
International SEO | | DohenyDrones0 -
Google Webmaster showing error for [hreflang='x-default']
Hi There! Using [hreflang='x-default'] tag to target language specific countries on our site but Google Webmaster showing errors even implementation made as per Google guideline but one thing is not clear and we are not sure, this is the reason behind it. Error is showing up only on those pages where 'Google Parameters' are used. For example : https://www.sitegeek.com/a2hosting?grank=open 'grank=' is defined as a 'Google Parameters' and on the above page 'hreflang' tags are : Also, on page https://www.sitegeek.com/a2hosting [without Google Parameters] same above 'hreflang' tags are taken. But, There is no error on second page URL where no 'Google Parameters' in URL. Therefore, error showing on first URL where 'Google Parameters' are taken. Is this the issue or not? suggest how to remove? -- Rajiv S9vhl3T
International SEO | | gamesecure0 -
Google is still indexing with https,i removed ssl for my website
My website is claydip.com. I removed ssl for my website, but when i type claydip in google search it is still displaying with https and saying no description available..i lost visitors from search..kindly help me. I moved from bluehost to deamhost. I had a ssl at bluehost, when i move to dreamhost i am not using it.
International SEO | | knextweb8190 -
Local SEO in Canada
I am trying to do some local optimization for some clients in Canada and it got me thinking, are there different best practices and different sites I want to use when working in Canada?
International SEO | | rbrianforrester0 -
How to replace my .co.uk site with my .com site in the US Google results
My customer and I are based in the UK. My customer's site, www.blindbolt.co.uk has been around for years. Last year we launched their American site, www.blindboltusa.com. Searching on google.com (tested both via proxy and using the gl=us querystring trick), a search for blind bolt on the US Google returns our www.blindbolt.co.uk site. We would like it to show our www.blindboltusa.com website in US searches. Webmaster tools has the Geographic Target set correctly for each site. Does anyone have any ideas or suggestions please? Thanks.
International SEO | | OffSightIT0 -
What’s the best way to convert ccTLD to global TLD?
We started out as a Canadian site targeting Canadian users. Now our site http://iCraft.ca has a lot of international buyers and sellers and .ca TLD doesn’t make sense anymore, as we are not performing well on Google.com We are doing a complete site redesign right now, which will address a lot of coding and content specific issues, but we suspect .ca domain will always hold us back in achieving good positions on Google.com. Since Google doesn’t allow ccTLDs to set geo-targeting, what are our options? a) Migrating to a brand new .com site and setting up 301 redirects for all links from iCraft.ca. Would we lose all rankings in this example and pretty much start building them from scratch? Or would PR be transferred page by page from one domain to another through 301 redirects? b) Setup a separate .com site with mirrored content to target global audience and keep .ca site to target Canada. Not sure if splitting PR for the same pages between 2 sites is a good idea. Also, how would you address duplicate content properly in our situation?
International SEO | | MarinaUX
In this video that I found here on forum http://www.youtube.com/watch?v=Ets7nHOV1Yo Matt Cutts says that it’s ok to have duplicate content on different ccTLDs, but he says - make sure you localize your content on those domains. What if you can’t? Most of the content on our site is meant for anyone, not just Canadian users. So, for the most part, we’d have exactly same content on .com site, as we have on .ca site. We could display prices in different currencies on product pages, but the rest of the content – blogs, forum etc. are not country-specific and can’t be localized easily. Also, it’s not clear from the video if all mirrored sites should sit on the same domain name for each country, like example.com and example.ca or is it ok to have example.com and icraft.ca? c) Is there a better option? Thanks for your help!0