International Sites - Sitemaps, Robots & Geolocating in WMT
-
Hi Guys,
I have a site that has now been launched in the US having originally just been UK. In order to accommodate this, the website has been set-up using directories for each country. Example:
As the site was originally set-up for UK, the sitemap, robots file & Webmaster Tools account were added to the main domain. Example:
The question is does this now need changing to make it specific for each country. Example:
The sitemap and robots.txt for the UK would move to:
and the US would have its own separate sitemap and robots.txt. Example :
Also in order to Geolocate this in WMT would this need to be done for each directory version instead of the main domain? Currently the WMT account for the UK site is verified at www.domain.com, would this need reverifying at domain.com/en-gb?
Any help would be appreciated! Thanks!
-
Thanks for the insights. Very helpful. What about the robots.txt, though? Should it stay under http://www.example.com, where crawlers can find the file?
-
Yes, that's how I did it.
-
Thanks for the response, much appreciated.
Would this mean that the sitemap featured at www.domain.com would need to reference each directory version (UK & US etc etc), leaving the en-gb & en-us to reference just their own?
-
Hi, here's what Google officially has to say:
Webmaster Tools data and reporting work best on a site level. For example, if your site http://www.example.com has separate sections for different countries, we recommend adding each of those subsites or subfolders as a separate site. For example, if you have a travel site with specific subfolders covering Ireland, France, and Spain, you could add the following sites to your Webmaster Tools account:
- http://www.example.com
- http://www.example.com/france
- http://www.example.com/ireland
- http://www.example.com/spain
Following this I have solved the exactly the same problem you had for a client. And it works perfectly.
- _The question is does this ["sitemap"] now need changing to make it specific for each country. _What I have done is add the sitemaps for each subdirectory as well as the main domain. It works (tracking and all that) and is easy to do.
- _Also in order to Geolocate this in WMT would this need to be done for each directory version instead of the main domain? _Yes, that is the way to go.
- _Currently the WMT account for the UK site is verified at www.domain.com, would this need reverifying at domain.com/en-gb? _Yes, it does. And so do every other subdirectory you add down the line. That allows precise international targeting per subdirectory via the hreflang and country targeting settings and setting the main GWT site for the root domain to be for global traffic.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ecommerce Product Page Optimization & International SEO
Hello, I'm working on our website SEO optimization. We have a thousands of products pages with different structures for the languages (arg) and very depth folder path .com/[folder]/[folder]/[folder]/product1.hmtl So now I have the happiness of working on the optimization of the website with themajor risk of impacting all current ranking. But anyway, here are a few questions I have on the way. Part 1 - International URL Our websites target people per country and languages. We do not have shops per countries (not enough resources_) but we try to get at least website per languages. What could be the best option?_ Url Parameters +hreflang So we save one folder less and the proper setup. But I'm just scared it's gonna be too messy for Google URL:.com/product1**?lang=fr** Product page:link rel="alternate" hreflang="en" href=".com/product1" / Language folder + hreflang one folder more but clearer structure URL:.com**/fr/**product1 **Product **page:****link rel="alternate" hreflang="en" href=".com/product1" / Part 2 - Product URL Our website is structure per categories so the product comes after. However, I've seen a lot of websites recently removing the categories to save folders space. What should be the most efficient option? Category folder It's obviously a good practice but this + the language folder makes already 2 folders URL:.com/categoryA/product1-{targetedKW} {targetedKW} = cheap product, best price or else All in url I've never done it but it somehow makes sense URL:.com/categoryA-product1-{targetedKW} Part 3 - Keyword stuffing As I'd like to get most of it automatically done, what could be the best places to add a few KW. **Markups:**All the ones we can **Meta Descriptions:**optimize one for Google + one for twitter + one for facebook Longer to do but then from google shopping and other automatic links, we could have the perfect or, at least, best description possible **All other option:**Reuse our product name + {targetter KW1 KW2 ...} Product description_ex: content_ Buttons (click to buy)ex: button title="Buy product_name cheap" alt="Purchase product_name"Buy Product name/button Images:same than above Meta:Titles and meta description Hn
International SEO | | omnyex0 -
Redirect to 'default' or English (/en) version of site?
Hi Moz Community! I'm trying to work through a thorny internationalization issue with the 'default' and English versions of our site. We have an international set-up of: www.domain.com (in english) www.domain.com/en www.domain.com/en-gb www.domain.com/fr-fr www.domain.com/de-de and so on... All the canonicals and HREFLANGs are set up, except the English language version is giving me pause. If you visit www.domain.com, all of the internal links on that page (due to the current way our cms works) point to www.domain.com/en/ versions of the pages. Content is identical between the two versions. The canonical on, say, www.domain.com/en/products points to www.domain.com/products. Feels like we're pulling in two different directions with our internationalization signals. Links go one way, canonical goes another. Three options I can see: Remove the /en/ version of the site. 301 all the /en versions of pages to /. Update the hreflangs to point the EN language users to the / version. **Redirect the / version of the site to /en. **The reverse of the above. **Keep both the /en and the / versions, update the links on / version. **Make it so that visitors to the / version of the site follow links that don't take them to the /en site. It feels like the /en version of the site is redundant and potentially sending confusing signals to search engines (it's currently a bit of a toss-up as to which version of a page ranks). I'm leaning toward removing the /en version and redirecting to the / version. It would be a big step as currently - due to the internal linking - about 40% of our traffic goes through the /en path. Anything to be aware of? Any recommendations or advice would be much appreciated.
International SEO | | MaxSydenham0 -
Correct site internationalization strategy
Hi, I'm working on the internationalization of a large website; the company wants to reach around 100 countries. I read this Google doc: https://support.google.com/webmasters/answer/182192?hl=en in order to design the strategy. The strategy is the following: For each market, I'll define a domain or subdomain with the next settings: Leave the mysitename.com for the biggest market in which it has been working for years, and define the geographic target in Google search console. Reserve the ccTLD domains for other markets In the markets where I'm not able to reserve the ccTLD domains, I'll use subdomains for the .com site, for example us.mysitename.com, and I'll define in Google search console the geographic target for this domain. Each domain will only be in the preferred language of each country (but the user will be able to change the language via cookies). The content will be similar in all markets of the same language, for example, in the .co.uk and in .us the texts will be the same, but the product selections will be specific for each market. Each URL will link to the same link in other countries via direct link and also via hreflang. The point of this is that all the link relevance that any of them gets, will be transmitted to all other sites. My questions are: Do you think that there are any possible problems with this strategy? Is it possible that I'll have problems with duplicate content? (like I said before, all domains will be assigned to a specific geographic target) Each site will have around 2.000.000 of URLs. Do you think that this could generate problems? It's possible that only primary and other important locations will have URLs with high quality external links and a decent TrustRank. Any other consideration or related experience with a similar process will be very appreciated as well. Sorry for all these questions, but I want to be really sure with this plan, since the company's growth is linked to this internationalization process. Thanks in advance!
International SEO | | robertorg0 -
Block access to site from everywhere but north america
I have a site that is being attacked very hard by bots, malware, etc. Most of it seems to be originating from Asia and Eastern Europe so I want to block off access to the site to everybody but people in North America. We do not ship out of the country anyways so it really does not need to be seen by people around the world. How can I set this up?
International SEO | | Atomicx0 -
What is the best way to generate an automatic sitemap for google, bing and yahoo?
Hi guys, I have 3 international sites So far I recieved a Sitemap that was generated by an seo company to use and submit to google, for our co.nz domain, I have been told to submit this also to bing and yahoo. Can anyone tell me if I can submit the same sitemap for the com.au and com or would i need to generate a new sitemap for each domain? I have been told, everytime we change content we have to keep submitting, is there a way to do this autmatically as we will be writting alot of content daily. Any recommendations or suggestions?
International SEO | | edward-may0 -
Duplicate Content - International Sites - AirBNB
Good morning Just a quick question. Why does AirBNB not get penalised for duplicate content on its sites. For example, the following two urls (and probably more for other countries), both rank appropriately in the google (UK and COM), https://www.airbnb.co.uk/help/getting-started/how-to-travel
International SEO | | joogla
https://www.airbnb.com/help/getting-started/how-to-travel Their are no canonical tags, no Alternative etc If I look at the following https://www.airbnb.co.uk/s/London--United-Kingdom
https://www.airbnb.com/s/London--United-Kingdom They both have alternative to point to the other language versions which I would expect. However they also both point to them selves as canonical. Would this not be duplicate content ? Thanks for your insights Shane0 -
IP Address Geolocation SEO - Multiple A records, implications?
Hi, We operate an ecom site, with a .com TLD. The IP address of the hosting is based in France and indeed we seem to see quite a lot of traffic from France. How relevant is the A record of the domain for SEO? Is it still an important signal to help Google geolocate? And, if that is the case, is there a case for having multiple A records for the domain? Like an IP Address in France, an IP address in Italy, etc... that way the domain would have multiple A records... Thank you
International SEO | | bjs20100 -
Geolocation Questions
I'm looking to combine my company's US web presence and its United Kingdoms web presence under one common look-feel and company name. Seeing as how we are fairly small, I'm thinking the best way to do this would be to simply create a "uk" folder and creating UK specific content in there. I would also like to have some geolocation on the site to make sure users receive the content that is relevant to them. With that in mind, here my questions: 1. Would creating a "locations" page with links between the UK and the US versions of the site, be enough so that Google is sure to crawl all content? (As I understand it, Google would appear as an American visitor to my geolocation script, and wouldn't see UK content unless there was a page that would explicitly direct it in that direction, correct?) 2. I've read elsewhere that I can target specific folders to a specific geographic target using Google Webmaster Tools. However, if the "main" site is US specific (there would not be a "us" folder) Setting the geographic target for JUST the "uk" folder would still work? 3. Finally, there will unfortunately be some duplicate content between the two sites. (we have a catalog of courses, for example, that contain different groupings of courses between the two sites, but the individual courses will appear with the same descriptions within the sites) What would be the best way to deal with something like that? I would hate to point all canonical links back to the US "main" site on every instance of duplicates, but I'm not sure how else to deal with it? Thanks for any help you can give. I know this is all a bit top level, but I'm a bit paralyzed with fear of starting, seeing as how I've never had to deal with these questions before...
International SEO | | TroyCarlson0