Google indexing https sites by default now, where's the Moz blog about it!
-
Hello and good morning / happy Friday!
Last night an article from of all places " Venture Beat " titled " Google Search starts indexing and letting users stream Android apps without matching web content " was sent to me, as I read this I got a bit giddy. Since we had just implemented a full sitewide https cert rather than a cart only ssl.
I then quickly searched for other sources to see if this was indeed true, and the writing on the walls seems to indicate so.
Google - Google Webmaster Blog! - http://googlewebmastercentral.blogspot.in/2015/12/indexing-https-pages-by-default.html
http://www.searchenginejournal.com/google-to-prioritize-the-indexing-of-https-pages/147179/
http://www.tomshardware.com/news/google-indexing-https-by-default,30781.html
https://hacked.com/google-will-begin-indexing-httpsencrypted-pages-default/
https://www.seroundtable.com/google-app-indexing-documentation-updated-21345.html
I found it a bit ironic to read about this on mostly unsecured sites.
I wanted to hear about the 8 keypoint rules that google will factor in when ranking / indexing https pages from now on, and see what you all felt about this.
Google will now begin to index HTTPS equivalents of HTTP web pages, even when the former don’t have any links to them. However, Google will only index an HTTPS URL if it follows these conditions:
- It doesn’t contain insecure dependencies.
- It isn’t blocked from crawling by robots.txt.
- It doesn’t redirect users to or through an insecure HTTP page.
- It doesn’t have a rel="canonical" link to the HTTP page.
- It doesn’t contain a noindex robots meta tag.
- It doesn’t have on-host outlinks to HTTP URLs.
- The sitemaps lists the HTTPS URL, or doesn’t list the HTTP version of the URL.
- The server has a valid TLS certificate.
One rule that confuses me a bit is :
- **It doesn’t redirect users to or through an insecure HTTP page. **
Does this mean if you just moved over to https from http your site won't pick up the https boost? Since most sites in general have http redirects to https?
Thank you!
-
Can you please make a concrete example of a key-word for that you do not rank nicely. Please also specify the thing which in your opinion need to appear nicely inside the serch and the object for the blog of nextgenapk .
-
Thanks for your response, Peter! As I said, I could be totally wrong - glad I asked this question
Cheers!
-
-
_"Or you can leave but change their links to pass some URL shortener - bit.ly or t.co until they comes with HTTPS version." _
looking at it from technical standpoint, these shortners are also not https (when crawling. Would they not have the same effect as other non https links?
Sorry, I could be going totally wrong about this and this question doesnt make sense at all.
-
Touche, good sir, these are certainly some great ways to go about this. Especially number 3.
Thanks!
Wonder how long we got until http2 implementation...
-
Or you can leave but change their links to pass some URL shortener - bit.ly or t.co until they comes with HTTPS version.
Or you can also make some page as "partners" where you can link only HTTP external sites.
Or you can also make internal page redirector to HTTP site. Like HTTPS -> HTTPS (inside redirector and dummy page) -> HTTP. On this case redirector won't be indexed and that's why it's dummy.
And this is just three ideas that i think for one minute. Probably mine favorite is #3. But it's IMHO.
-
So if my manufacturers don't have https sites, I should remove the links to them since it's going to hinder indexing?
Thanks for the http redirecting to https response.
-
Some sites comes with redirectors or "beacons" for detecting user presence. Example i'm on site X page A and there i click on link to go on page B. But due marketing department this pass via HTTP redirector or pure HTTP (and there 301 redirect to HTTPS). Then this page B can be not indexed.
This mean that once you set sitewide 301 redirect to encrypted connection you must make few more steps:
- you must check all resources to pass via this encrypted channel. Images, CSS, JS - just anything.
- you must check canonical to be set to HTTPS
- you must check that link between pages to be also HTTPS
- you must see any 3rd party tools for encrypted connection. Can be analytics software or "tracking pixels" or heat maps or ads.
- you must check if outgoing links from your site can be via other sites with encryption. Can be Wikipedia, Moz, Google. Since everything there is already encrypted you will skip frustrating HTTPS -> HTTP -> HTTPS jump too.
So then your site can be indexed in HTTPS. It's tricky procedure with many traps.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Search traffic plummeting after HTTPS fumble - what to do now?
Hi all, Our website typically gets about 80% of our traffic from organic Google search over thousands of keywords (i.e., no single keyword (or group of) drives a large portion of our traffic). It's a nine year old website, and we have been growing steadily -- including about 30-40% year-over-year growth for the past 9-months. That is, up until Feb 2nd. On February 2nd, we switched to HTTPS. Everything was done per Google's recommendations: pages individually 301'd to HTTPS pages, no security warnings, added the new site in Webmaster Tools, etc. Google started to pick up our new site -- albeit 3 weeks into the transition, traffic was still significantly down. However, the big problem that we discovered was our ad revenues were getting destroyed. We're an ad based business and our CPMs were tanking, some of our ad partners were having problems serving ads, etc. We were losing a lot of money. So, we made the decision to reverse the HTTPS change and go back to HTTP. That was on Feb 22nd. Our traffic started to recover, and our ad rates did recover. However, 2-weeks after switching back -- March 8 -- our traffic started to fall and has continued to do so. Our traffic is now half of what it was a year ago, and only 1/3 of what it was before we made any changes. I am totally at a loss for what to do. I have spent endless hours digging through Webmaster Tools with no real insights. Here's the most I've been able to glean: Google picked up the new HTTPS site a lot faster than it has reverted back to the HTTP. Particularly for AMP pages. We had about 2,000 indexed AMP pages, which were quickly picked up when we switched to HTTPS, but since changing back to HTTP Google has been slow to re-index the HTTP. Only 935 AMP indexed pages now. According to Webmaster Tools, our overall ranking position has not been affected (the overall average). However, in a sampling of keywords I notice that a number of keywords seem to have been dropped completely from ranking, while others show the same rank position but Google seems to only be showing us in the results intermittently -- e.g., rank is unchanged, but impressions and clicks are much lower. I do not know what to do at this point, and sadly, I'm starting to get desperate for some help. I feel like all the hard work of almost a decade is slipping away and I have no idea how to change course. I've done absolutely everything I can think of from a technical standpoint. Am I being penalized for abandoning the switch to HTTPS? Should I now try and reverse course again, and switch BACK to HTTPS? Is this a temporary bobble that Google's algo will 'forget'? It's a super high quality website with long, unique, detailed articles. Not spammy and we have never had a manual action against us. I don't know what to do. Please help! Here's a link to the website. Thank you in advance.
Algorithm Updates | | tustind0 -
Weekly traffic from Google: How do you explain this?
Hello here, I have question for you. Please, have a look at the attached image from my website analytics which shows the unique visits trend of the last 2 months. What it is interesting is that every Monday Google brings me more traffic than any other day of the week, whereas on Saturdays it gives me the lowest traffic. And looks like that's a pretty regular weekly pattern. Why is Google doing that? What does that mean? Why such a clear and steady pattern? I am eager to know your thoughts about this! CGuULrN.jpg
Algorithm Updates | | fablau0 -
Google is forcing a 301 by truncating our URLs
Just recently we noticed that google has indexed truncated urls for many of our pages that get 301'd to the correct page. For example, we have:
Algorithm Updates | | mmac
http://www.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html as the url linked everywhere and that's the only version of that page that we use. Google somehow figured out that it would still go to the right place via 301 if they removed the html filename from the end, so they indexed just: http://www.eventective.com/USA/Massachusetts/Bedford/107/ The 301 is not new. It used to 404, but (probably 5 years ago) we saw a few links come in with the html file missing on similar urls so we decided to 301 them instead thinking it would be helpful. We've preferred the longer version because it has the name in it and users that pay attention to the url can feel more confident they are going to the right place. We've always used the full (longer) url and google used to index them all that way, but just recently we noticed about 1/2 of our urls have been converted to the shorter version in the SERPs. These shortened urls take the user to the right page via 301, so it isn't a case of the user landing in the wrong place, but over 100,000 301s may not be so good. You can look at: site:www.eventective.com/usa/massachusetts/bedford/ and you'll noticed all of the urls to businesses at the top of the listings go to the truncated version, but toward the bottom they have the full url. Can you explain to me why google would index a page that is 301'd to the right page and has been for years? I have a lot of thoughts on why they would do this and even more ideas on how we could build our urls better, but I'd really like to hear from some people that aren't quite as close to it as I am. One small detail that shouldn't affect this, but I'll mention it anyway, is that we have a mobile site with the same url pattern. http://m.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html We did not have the proper 301 in place on the m. site until the end of last week. I'm pretty sure it will be asked, so I'll also mention we have the rel=alternate/canonical set up between the www and m sites. I'm also interested in any thoughts on how this may affect rankings since we seem to have been hit by something toward the end of last week. Don't hesitate to mention anything else you see that may have triggered whatever may have hit us. Thank you,
Michael0 -
Regarding site url structure
OK so there are already some answers to questions similar to this but mine might be a little more specific. OK website is www.bestlifeint.com Most of our product pages are as such: http://www.bestlifeint.com/products-soy.html for instance. However I was trying to help the SEO for certain pages (namely two) with the URL's and had some success with another page our Soy Meal Replacement I changed the site URL of this page from www.bestlifeint.com/products-meal to www.bestlifeint.com/Soy-Amazing-Meal-Replacement-with-Omega-3s.html (notice I dropped the /product part of url and made it more seo friendly. The old page for this page was something like www.bestlifeint.com/products-meal The issue is that recently this new page and another page I have changed http://www.bestlifeint.com/Whey-Milk-Alternative.html I have dropped the "/product" on the URL even though they are both products. The new Meal Replacement page used to be ranked like 6th on google at the begining of the month and now is like 48th or something. The new "whey milk" page (http://www.bestlifeint.com/Whey-Milk-Alternative.html) is ranked like 45th or something for "Whey Milk" when the old page...."products/wheyrice.html" was ranked around 18th or so at the begining of the month. Have I hurt these two pages by not following www.bestlifeint.com/product.... site structure? And focusing more on the URL SEO? I have both NEW pages receiving all link juice inside web site so they are the new pages (can not go to old page) and recently seeing that google has pretty much dropped the old pages in search rankings I have deleted these two pages. Do i just need to just wait and see? According to my research we should rank much higher for "Whey Milk" we should be on the first page according to googles own statements of searchers finding good relevant material. Any advice moving forward? Thanks, Brian
Algorithm Updates | | SammisBest0 -
How vital is it for a site to have a mobile site for mobile SEO?
With the exponential growth in mobile device sales and usage and an expected 980% growth in advertising next year for/on mobile devices, we at http://www.mobilewebsitegurus.com decided that it was time to help companies create great looking mobile websites that are user friendly and SEO friendly at affordable rates with tons of features built in from the start. However, when selling our design, how important is it to have a GOOD mobile site compared to a big one to rank on mobile devices? We head that Google was thinking of only showing mobile sites on mobile devices. NOT TRUE. Then we read/heard that the rankings were MUCH BETTER if you had a mobile site, but after a lot of research we found that too NOT to be true. On most sites there were NO difference. So what is the TRUTH about this and is it maybe just that it will happen, just has not happened yet - the different rankings for mobile and regular sites on mobile devices that is? ANY insight in this would be great not only for us but for the entire SEO community 🙂 Thanks. ALSO, add "Mobile SEO" to the boxes below of "Topics" since mobile SEO will grow in importance.
Algorithm Updates | | yvonneq0 -
Trying to figure out why one of my popular pages was de-indexed from Google.
I wanted to share this with everyone for two reasons. 1. To try to figure out why this happened, and 2 Let everyone be aware of this so you can check some of your pages if needed. Someone on Facebook asked me a question that I knew I had answered in this post. I couldn't remember what the url was, so I googled some of the terms I knew was in the page, and the page didn't show up. I did some more searches and found out that the entire page was missing from Google. This page has a good number of shares, comments, Facebook likes, etc (ie: social signals) and there is certainly no black / gray hat techniques being used on my site. This page received a decent amount of organic traffic as well. I'm not sure when the page was de-indexed, and wouldn't have even known if I had't tried to search for it via google; which makes me concerned that perhaps other pages are being de-indexed. It also concerns me that I have done something wrong (without knowing) and perhaps other pages on my site are going to be penalized as well. Does anyone have any idea why this page would be de-indexed? It sure seems like all the signals are there to show Google this page is unique and valuable. Interested to hear some of your thoughts on this. Thanks
Algorithm Updates | | NoahsDad0 -
Google Cached Pages
I made some on-site changes to a site last week, in particular their page titles. This was all done on the same day at the same time. Now, one of those pages, got re-indexed on August 8th and has my updated changes, which also helped with my ranking. The other page I made changes to still shows a cached version from July 27th, which is before I made the changes. Why wouldn't google have an updated page from August 8th for both pages, not just one?
Algorithm Updates | | MichaelWeisbaum0 -
Google Update on the 6th July
Hi Mozzers, Has anyone noticed a Google update on the 6th July? A price comparison site I optimise has fallen off the SERPs for most generic terms, however still getting traffic for longer tail phrases. Cheers Aran
Algorithm Updates | | Entrusteddev0