Geolocation and Indexing
-
Hi all,
Our company owns site that have over 5 millions pages in Google index. We are locating in German, but our business aimed to US market.
So, recently I checked index of our site using region targeting in US and there were only 150k of pages, but when I checked targeting in German there were almost 5 billion pages.
Our server/IP locating in US, all the backlinks are from US sites.
So, why there it is only small part of the site indexed in US?
Regards,
Dmitry
-
What exact do you mean by "when I checked targeting in German there were almost 5 billion pages"? I know you are referring to "million" but how did you arrive at that number?
Other things to check:
-
submit an updated sitemap to Google. How many pages show in the site map?
-
what type of navigation does your site offer? Is all of the navigation visible in HTML?
-
some sites offer dozens of versions of the same page. A print-friendly version, sorted ascending by price, sort descending by price, sort by size and many other properties. Each sort is a different page on your site. You can have a site with 150k worth of canonical pages, but 5 million actual pages. Google will not list the duplicate pages.
-
-
Sorry, that was my mistake, I meant 5 million pages.
Unfortunately I can't name the domain name. Our site is 100% US based, with English content. I'm asking for is there some other issues (not Panda, content, etc.), that can cause the situation with regional indexation.
-
Our company owns site that have over 5 billion pages in Google index.
In order to help you, some specifics would be needed. What is the URL of the site?
Off the top of my head I would think Amazon.com is one of the biggest sites around and they have around 320 million pages indexed. The largest forum site in the world has about 16 million pages indexed by Google.
The only site I can think of with billions of indexed pages would be a scraper or other form of content manipulation website.
You mentioned you are located in Germany so clearly your pages are going to be considered most relevant there. If you wish to be more relevant to US sites, the content would need to be presented in US English, use English measurements, currency, references, etc. You would desire links from US sites as well. You could go into Google WMT and set US as your preferred country, but that would mean you would lose a significant amount of your German indexing.
Also consider the US has fully implemented Panda. It is coming to Germany but has not been implemented there yet (to the best of my knowledge, I could be mistaken). If you have a billion or more pages, I am going to speculate a huge percentage of pages are duplicated both internally to your site, and externally to the internet. If that is the case, the number of indexed pages will take a huge hit.
If your site is deemed untrustworthy due to scraped content, your entire site may be de-indexed until the issue is resolved.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hreflang tags and canonical tags - might be causing indexing and duplicate content issues
Hi, Let's say I have a site located at https://www.example.com, and also have subdirectories setup for different languages. For example: https://www.example.com/es_ES/ https://www.example.com/fr_FR/ https://www.example.com/it_IT/ My Spanish version currently has the following hreflang tags and canonical tag implemented: My robots.txt file is blocking all of my language subdirectories. For example: User-agent:* Disallow: /es_ES/ Disallow: /fr_FR/ Disallow: /it_IT/ This setup doesn't seem right. I don't think I should be blocking the language-specific subdirectories via robots.txt What are your thoughts? Does my hreflang tag and canonical tag implementation look correct to you? Should I be doing this differently? I would greatly appreciate your feedback and/or suggestions.
International SEO | | Avid_Demand0 -
My indexed site URL removed from google search without get any message or Manual Actions??
On Agust 2 or 3.. I'm not sure about the exact date...
International SEO | | newwaves
The main URL of my website https://new-waves.net/ had been completely removed from Google search results! without getting any messages or Manual Actions on search console ?? but I'm still can find some of my site subpages in search results and on Google local maps results when I tried to check it on google
info:new-waves.net >> no results
site:new-waves.net >> only now I can see the main URL in results because I had submitted it again and again to google but it might be deleted again today or tomorrow as that happen before last few days
100% of all ranked keywords >> my site URL new-waves.net had been completely removed from all results! but I'm still can see it on maps on some results My site was ranked number 1 on google search results for "digital marketing qatar" and some other keywords, but the main URL had been removed from 100% of all search results. but you can still see it on the map only. I just tried to submit it again to Google and to index it through google search console tool but still not get any results, Can any one help to know what is the reason?? and how can I solve this issue without losing my previous ranked keywords? Can I submit a direct message to google support or customer service to know the reason or get help on this issue? Thanks & Regards0 -
Geolocation issue: Google not displaying the correct url in the SERP's
Hello, Im running a multi-country domain with this structure: domain.com/ar/
International SEO | | EstebanCervi
domain.com/mx/
domain.com/cl/
etc I also have: domain.com/int/ for x-default
domain.com/category/ does a 301 redirect through IP geo-location to the correspondent url, example if your IP is from Mexico, then you got redirected to domain.com/mx/category/ hreflang is correct. webmaster tool geo-location is correct. Example of the issue Im facing right now: When users from Chile do a keyword search in Google Chile, the domain ranks well but the URL that appears in the SERP is the /mx/ version, or the /int/ version or any other country version. Other times is the /cl/ version. The same happens for all the users / countries / keywords. I need to understand what Im doing wrong, because Google is not displaying in the SERP's the correct URL version for the country of the user who is doing the search. Thank you so much! I will appreciate your ideas. PS: I think I should try to change the 301 to a 302 redirect, or completely remove those redirects. Any ideas? Suggestions? Thanks!0 -
Massive jump in pages indexed (and I do mean massive)
Hello mozzers, I have been working in SEO for a number of years but never seen anything like a jump in pages indexed of this proportion (image is from the Index Status report in Google Webmaster Tools: http://i.imgur.com/79mW6Jl.png Has anyone has ever seen anything like this?
International SEO | | Lina-iWeb
Anyone have an idea about what happened? One thing that sprung to mind might be that the same pages are now getting indexed in several more google country sites (e.g. google.ca, google.co.uk, google.es, google.com.mx) but I don't know if the Index Status report in WMT works like that. A few notes to explain the context: It's an eCommerce website with service pages and around 9 different pages listing products. The site is small - only around 100 pages across three languages 1.5 months ago we migrated from three language subdomains to a single sub-domain with language directories. Before and after the migration I used hreflang tags across the board. We saw about 50% uplift in traffic from unbranded organic terms after the migration (although on day one it was more like +300%), especially from more language diversity. I had an issue where the 'sort' links on the product tables were giving rise to thousands of pages of duplicate content, although I had used the URL parameter handling to communicate to Google that these were not significantly different and only to index the representative URL. About 2 weeks ago I blocked them using the robots.txt (Disallow: *?sort). I never felt these were doing us too much harm in reality although many of them are indexed and can be found with a site:xxx.com search. At the same time as adding *?sort to the robots.txt, I made an hreflang sitemap for each language, and linked to them from an index sitemap and added these to WMT. I added some country specific alternate URLs as well as language just to see if I started getting more traffic from those countries (e.g. xxx.com/es/ for Spanish, xxx.com/es/ for Spain, xxx.xom/es/ for Mexico etc). I dodn't seem to get any benefit from this. Webmaster tools profile is for a URL that is the root domain xxx.com. We have a lot of other subdomains, including a blog that is far bigger than our main site. But looking at the Search Queries report, all the pages listed are on the core website so I don't think it is the blog pages etc. I have seen a couple of good days in terms of unbranded organic search referrals - no spike or drop off but a couple of good days in keeping with recent improvements in these kinds of referrals. We have some software mirror sub domains that are duplicated across two website: xxx.mirror.xxx.com and xxx.mirror.xxx.ca. Many of these don't even have sections and Google seemed to be handling the duplication, always preferring to show the .com URL despite no cross-site canonicals in place. Very interesting, I'm sure you will agree! THANKS FOR READING! 79mW6Jl.png0 -
Robots.txt issue with indexation
Hello i have a problem with one of the rules for robots.txt i have a multilingual mutation of entire page on www.example.com/en/ I want to make indexable /allow/ the main page under /en/ but not indexable /disallow/ everything else under /en/* Please help me how to write the rule.
International SEO | | profesia0 -
Non US site pages indexed in US Google search
Hi, We are having a global site wide issue with non US site pages being indexed by Google and served up in US search results. Conversley, we have US en pages showing in the Japan Google search results. We currently us IP detect to direct users to the correct regional site but it isn't effective if the users are entering through an incorrect regional page. At the top of each or our pages we have a drop down menu to allow users to manually select their preferred region. Is it possible that Google Bot is crawling these links and indexing these other regional pages as US and not detecting it due to our URL structure? Below are examples of two of our URLs for reference - one from Canada, the other from the US /ca/en/prod4130078/2500058/catalog50008/ /us/en/prod4130078/2500058/catalog20038/ If that is, in fact, what is happening, would setting the links within the drop down to 'no follow' address the problem? Thank you. Angie
International SEO | | Corel0 -
Geolocation Questions
I'm looking to combine my company's US web presence and its United Kingdoms web presence under one common look-feel and company name. Seeing as how we are fairly small, I'm thinking the best way to do this would be to simply create a "uk" folder and creating UK specific content in there. I would also like to have some geolocation on the site to make sure users receive the content that is relevant to them. With that in mind, here my questions: 1. Would creating a "locations" page with links between the UK and the US versions of the site, be enough so that Google is sure to crawl all content? (As I understand it, Google would appear as an American visitor to my geolocation script, and wouldn't see UK content unless there was a page that would explicitly direct it in that direction, correct?) 2. I've read elsewhere that I can target specific folders to a specific geographic target using Google Webmaster Tools. However, if the "main" site is US specific (there would not be a "us" folder) Setting the geographic target for JUST the "uk" folder would still work? 3. Finally, there will unfortunately be some duplicate content between the two sites. (we have a catalog of courses, for example, that contain different groupings of courses between the two sites, but the individual courses will appear with the same descriptions within the sites) What would be the best way to deal with something like that? I would hate to point all canonical links back to the US "main" site on every instance of duplicates, but I'm not sure how else to deal with it? Thanks for any help you can give. I know this is all a bit top level, but I'm a bit paralyzed with fear of starting, seeing as how I've never had to deal with these questions before...
International SEO | | TroyCarlson0 -
De-Indexing URLs from a specific Locale
Is it possible to de-index a specific URL from showing up in a specific locale? For example, if I want to de-index http://www.example.com/category/product1 from http://www.google.co.uk but not http://www.google.com, is that possible?
International SEO | | craigsmith3330