International Sites and Duplicate Content
-
Hello,
I am working on a project where have some doubts regarding the structure of international sites and multi languages.Website is in the fashion industry. I think is a common problem for this industry. Website is translated in 5 languages and sell in 21 countries.
As you can imagine this create a huge number of urls, so much that with ScreamingFrog I cant even complete the crawling.
Perhaps the UK site is visible in all those versions
http://www.MyDomain.com/en/GB/
http://www.MyDomain.com/it/GB/
http://www.MyDomain.com/fr/GB/
http://www.MyDomain.com/de/GB/
http://www.MyDomain.com/es/GB/
Obviously for SEO only the first version is important
One other example, the French site is available in 5 languages and again...
http://www.MyDomain.com/fr/FR/
http://www.MyDomain.com/en/FR/
http://www.MyDomain.com/it/FR/
http://www.MyDomain.com/de/FR/
http://www.MyDomain.com/es/FR/
And so on...this is creating 3 issues mainly:
-
Endless crawling - with crawlers not focusing on most important pages
-
Duplication of content
-
Wrong GEO urls ranking in Google
I have already implemented href lang but didn't noticed any improvements. Therefore my question is
Should I exclude with "robots.txt" and "no index" the non appropriate targeting?
Perhaps for UK leave crawable just English version i.e. http://www.MyDomain.com/en/GB/, for France just the French version http://www.MyDomain.com/fr/FR/ and so on
What I would like to get doing this is to have the crawlers more focused on the important SEO pages, avoid content duplication and wrong urls rankings on local Google
Please comment
-
-
Hey Guido, don't know if it's the best solution, but could be a temporary fix until the best solution is in place. I suggest to move forward with proper HREF LANG tagging or definitely delete those irrelevant languages. Try to do what I said before about validate each country/language and submit a sitemap.xml reflecting that folder to see crawl and index stats pero country/language. Add a sitemap index and obviously validate your entire domain. Just block in the robots.txt unnecessary folders, like images, js libraries, etc. to save crawl budget to your domain.
Let me know if you have another doubt
-
Thank you Antonio, insightful and clear.
There is really not a need of EN versions of localized sites, I think has been done more as was easier to implement (original site is EN-US).
Don't you think robots and noindex EN version of localized sites could be the best solution? for sure is the easier one to implement without affecting UX.
-
Don't know why you have a UK oriented site for German and Italian people, I think is not important those languages in a country mainly English speaking (not US for example, there you must have a Spanish version, or in Canada for English and French). The owner must have their reasons.
Besides this, about your questions:
- If those non-relevant languages must live there, it's correct to implement HREF LANG (may take some time to show results). Also, if the domain is gTLD, you can validate all the subfolders in Google Search Console and choose the proper International targeting. With the ammount of languages and countries I imagine this might be a pain in the ***.
- About the crawling, for large sitesI recommend to crawl per language. If neccesary, per language-country. In this instance I recommend to create a sitemap XML per language or language-country for just HTML pages (hopefully dynamically updated by the e-commerce), create a Sitemap Index in the root of the domain and submit them in Google Search Console (better if you validated the languages or language-country). With this you can answer the question if some language or country are being not indexed with the Submited/Indexed stadistics of GSC.
- Maybe the robots.txt might save your crawl budget, but I'm not a fan of de-index if those folders are truly not relevant (after all, there should be a italian living in UK. If you can't delete the irrelevant langauges for some countries, this can be an option
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
International SEO
Okay, so I have read through the following link in respect to International SEO (https://mza.bundledseo.com/learn/seo/international-seo), and I believe that the way forward it a ccTLD. My thought was to have .com, .co.uk and .eu. Currently my site is .com, but receives most of its traffic from UK sources. I'm concerned that when I switch over to ccTLDs, the .co.uk in particular, that my UK traffic could dry up. Switching from .com to .co.uk and then using the .com to target the US market makes sense, but I would like to know others opinions on the potential dangers of doing this. Also, are ccTLDs kept on the same hosting or would they require individual hosting? The link doesn't cover this question.
International SEO | | moon-boots1 -
How to deal with disproportional content investment for a ccTLD for a multi-language country,
We have a website for the Belgium market, serving content and products on be/nl (Dutch/Flemmish Belgium) and .be/fr (French Belgium). However, as a Dutch-based company you can see our primary focus and objective is to serve content to Dutch Belgium rather than French Belgium. I wonder if, and so, what are the downsides are of only investing in half of the site?
International SEO | | Marketing-Omoda
Does it hurt my general .be Google rankings if we put a lot of effort in .be/nl but far less in .be/fr ? (we used to have a ccTLD .fr as well, but pulled the plug because it wasn't profitable.
our belgium website is profitable for Dutch speaking part of Belgium but now we would like to expand, and enhance rankings. We're investing heavily in (local) brand awareness and partnerships, and content marketing for the Dutch part.0 -
Can multiple hreflang tags point to one URL? International SEO question
Moz, Hi Moz, Can multiple hreflang tags point to a single URL? For example, if I have a Canadian site (www.example.com/ca) that targets French and English speakers can I have the following: or would I use: Any insight would be very helpful and greatly appreciated! Thank you in advance!
International SEO | | DA20131 -
International SEO Subfolders / user journey etc
Hi According to all the resources i can find on Moz and elsewhere re int seo, say in the context of having duplicate versions of US & UK site, its best to have subfolders i.e. domain.com/en-gb/ & domain.com/en-us/ however when it comes to the user journey and promoting web address seems a bit weird to say visit us at: domain.com/en-us/ !? And what happens if someone just enters in domain.com from the US or UK ? My client wants to use an IP sniffer but i've read thats bad practice and should employ above style country/language code instead, but i'm confused about both the user journey and experience in the case of multiple sub folders. Any advice much appreciated ? Cheers Dan
International SEO | | Dan-Lawrence0 -
International websites : hreflang
Hi, i'm looking for good examples with 'href lang' tag (rel="alternate" hreflang="x") Have you examples of websites with this tag? Thanks D.
International SEO | | android_lyon0 -
Duplicate content or not ?
Hello, I would like your expert opinion I have a site in spanish for Spain and Mexico As domain name, I have .es and .mx This is the same site. We do not have any redirects. From .mx to .es for example. >> your opinion?
International SEO | | android_lyon
if I declare targeting in Spain in Google Webmaster tools (in settings) and in another profile with in Mexico, we have a duplicate content? Thank you for your feedback. Sorry for my english, i'm french 😉0 -
Looking for content writers for multi-language SEO
Hi All, I'm currently doing a lot of work for a UK client who has multiple sites outside the UK (all part of the same business). We're currently discussing the option of us handling all of his SEO for his German, French, Spanish and Italian sites too, but we only have access to one person in the office who can speak French and Spanish. They're currently booked up on other jobs that we can't really move them off, so I'm looking for options of outsourcing some of the content writing. My question is, does anyone know of any high quality content writing services that have writers available to write for the countries languages above? We're going to focus initially on their on-site strategy and building up their high quality content. At the moment, they don't have much relevant content on their website, so we're going to initially look at this. Moving forward, we'll be looking at their off-site strategy and trying to find areas to submit high quality articles, look at guest blogging and PR opportunities. Any tips anyone has on this side (in terms of outsourcing to native speakers) would be quite useful too! Many thanks,
International SEO | | PinpointDesigns
Lewis0 -
Impact of Japanese .jp site duplicate content?
Our main website is at http://www.traxnyc.com and we just launched a Japanese version of the site at http://www.traxnyc.jp domain. However all the images used on the .jp site are linked from the .com site. Would this hurt me in Google at all for hotlinking images? Also there is quite a bit of duplicate content on the .jp site at the moment: only a few things have been translated to Japanese and for the most part the layouts and words are exactly the same (in English). Would this hurt my Google rankings in the US at all? Thanks for all your help.
International SEO | | DiamondJewelryEmpire0