HTTP Vary:User-Agent Server or Page Level?
-
Looking for any insights regarding the usage of the Vary HTTP Header. Mainly around the idea that search engines will not like having a Vary HTTP Header on pages that don't have a mobile version, which means the header will be to be implemented on a page-by-page basis.
Additionally, does anyone has experience with the usage of the Vary HTTP Header and CDNs like Akamai?Google still recommends using the header, even though it can present some challenges with CDNs.
Thanks!
-
hey burnseo - if you're still getting notifications from this thread, would you happen to recall where you ended up finding info. that google recommends placing the vary header at page level? running into the same question myself. if you have links you could post to where you found the answer, that'd be great. thanks!
-
I would go by what Google recommends I cannot imagine Akamai being something bad for website or overwhelming it anyway. You may try using a C name with your www. straight to the CDN & if you're using a mobile subdomain like m. also having that go directly into your content delivery network.
I hope this is better help.
sincerely,
Thomas
-
I found some information that suggests that it is recommended to avoid using the Vary HTTP Header by User-Agent site-wide because search engines and (and this is Google) would assume the other version simply hadn't yet been discovered and perhaps keep looking for it. There is also a recommendation to implement the Vary Header on a page-level only when there is a mobile version. This only applies to sites that are serving mobile HTML content dynamically based in the user-agent. Additionally, there is some controversy around using the header when a CDN network like Akamai is in place because it can overload the site. Despite this controversy Google still recommends using the header. These seem to be two important points to consider before implementing the Vary HTTP Header.
-
Very true I shoud have compleated it woun't use a cell phone to Q&A
-
Thomas, it appears that this is taken from http://stackoverflow.com/questions/1975416/trying-to-understand-the-vary-http-header. Q&A is for original answers; if you are referring to another blog post, it's best to just put a link into the blog post and let people go there rather than copy work (that may be copyright) and use that as your answer. Thanks for understanding!
-
-
The
cache-control
header is the primary mechanism for an HTTP server to tell a caching proxy the "freshness" of a response. (i.e., how/if long to store the response in the cache) -
In some situations,
cache-control
directives are insufficient. A discussion from the HTTP working group is archived here, describing a page that changes only with language. This is not the correct use case for the vary header, but the context is valuable for our discussion. (Although I believe the Vary header would solve the problem in that case, there is a Better Way.) From that page:
Vary
is strictly for those cases where it's hopeless or excessively complicated for a proxy to replicate what the server would do.- This page describes the header usage from the server perspective, this one from a caching proxy perspective. It's intended to specify a set of HTTP request headers that determine uniqueness of a request.
A contrived example:
Your HTTP server has a large landing page. You have two slightly different pages with the same URL, depending if the user has been there before. You distinguish between requests and a user's "visit count" based on Cookies. But -- since your server's landing page is so large, you want intermediary proxies to cache the response if possible.
The URL, Last-Modified and Cache-Control headers are insufficient to give this insight to a caching proxy, but if you add
Vary: Cookie
, the cache engine will add the Cookie header to it's caching decisions.Finally, for small traffic, dynamic web sites -- I have always found the simple
Cache-Control: no-cache, no-store
andPragma: no-cache
sufficient.Edit -- to more precisely answer your question: the HTTP request header 'Accept' defines the Content-Types a client can process. If you have two copies of the same content at the same URL, differing only in Content-Type, then using
Vary: Accept
could be appropriate.Update 11 Sep 12:
I'm including a couple links that have appeared in the comments since this comment was originally posted. They're both excellent resources for real-world examples (and problems) with Vary: Accept; Iif you're reading this answer you need to read those links as well.
The first, from the outstanding EricLaw, on Internet Explorer's behavior with the Vary header and some of the challenges it presents to developers: Vary Header Prevents Caching in IE. In short, IE (pre IE9) does not cache any content that uses the Vary header because the request cache does not include HTTP Request headers. EricLaw (Eric Lawrence in the real world) is a Program Manager on the IE team.
The second is from Eran Medan, and is an on-going discussion of Vary-related unexpected behavior in Chrome:Backing doesn't handle Vary header correctly. It's related to IE's behavior, except the Chrome devs took a different approach -- though it doesn't appear to have been a deliberate choice.
-
-
Hey Thomas, thank you for your interest in answering my question. However, the question isn't really about using a CDN. It is more around how using the Vary HTTP Header can affect the CDN performance. In addition, I wanted to find guidance on where to implement the Vary HTTP Header as it was brought to my attention that search engines don't like it when this is implemented site wide even on pages that don't have a mobile version.
-
Hi Keri,
Thank you for the heads up on that. I definitely was having some technical issues. I have cleaned it up let me know if you think it is a need any more work.
Thank you for letting me know.
Sincerely,
Thomas
-
Thomas, I think the voice recognition software botched some of your reply. Could you go through and edit it a little? There are some words that seem to be missing. Thanks!
-
Hi,
For insights regarding the usage of the Vary HTTP Header.
I would check out this blog post right here
As far as using a content delivery network. I love them and have used quite a few. Depending on your budget there is a wide range
Use Anycast DNS with CDN's here is what I think of them.
#1 DNS DynECT (my fav)
#2 DNS Made Easy (great deal $25 for 10 domains for the YEAR)
#3 UltraDNS
#4 VerisignDNS
CDN's many have anycast DNS built in already
Check out this website it will give you a good view of what's going on this site
http://www.cdnplanet.com/cdns/
I don't know what you want for data however if you want a great CDN with support & killer price Max CDN it's only $39 for the first terabyte performs Amazon's cloudflaire Rackspace clouldfiles
My list of CDN's I would use the cost is anywhere form $39 a year to $4,000 a month if you said you where going to use video it will cost more as data adds up fast.
#1 Level 3 personal favorite content delivery network
http://www.level3.com/en/products-and-services/data-and-internet/cdn-content-delivery-network/
http://www.edgecast.com/free-trial/
http://mediatemple.net/webhosting/procdn/ You get 200 gb's a month for $20 it is 100% EdgeCast (just a reseller)
https://presscdn.com/ PRESSCDN is 50GB's for $10 month & gives you FOUR CDN's it has Max CDN, Edgecast, Akamai & cloudfront price for 150GB a month is $19
http://www.rackspace.com/cloud/files/
http://aws.amazon.com/cloudfront/
Look a thttp://cloudharmony.com/speedtest for speed testing
However please remember that coding makes a huge difference on websites and it is not really a fair depiction of speed.
You could use CloudFlare it is free I don't like it for for anything other than site protection it's not very fast and my opinion and is simply a proxy reverse proxy server
You get CloudFlare with Railgun already on
https://www.cloudflare.com/railgun cost is now $200 a month (Use Level 3 if paying that much)
Edge cast is a great content delivery network. However,you will have to buy it through a third-party that you want a full enterprise version. You can buy to media temple that you must use their DNS and it is only $20 a month.
However if you're going to spend over $20 a month I would strongly consider talking to Level 3. There notoriously high-priced however they just lowered their prices and you can negotiate some very sweet deals.
I would simply sign up for DNS made easy and MaxCDN if you don't have a content delivery network already & just convenient fast
It's also faster. It is faster than AWS cloudfront & rack space cloudfiles.
Max CDN is faster than anything else I have compared to the it's price range for almost double
But inexpensive service you will get Anycast DNS for $25 and the CDN would be $39 and that's for the year not the month
I hope this is been of help to you,and it answers your question. Please let me know if I could be of any more help.
Sincerely,
Thomas
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Thousands of 404-pages, duplicate content pages, temporary redirect
Hi, i take over the SEO of a quite large e-commerce-site. After checking crawl issues, there seems to be +3000 4xx client errors, +3000 duplicate content issues and +35000 temporary redirects. I'm quite desperate regarding these results. What would be the most effective way to handle that. It's a magento shop. I'm grateful for any kind of help! Thx,
Technical SEO | | posthumus
boris0 -
How to make my good sub-page rank ahead of my generic home page?
I have an ecommerce site for the clothes drying racks my family business makes, and it sells a few other laundry items also. It's about 5 years old. We used to rank on the first page for basic phrases like "clothes drying rack" and "umbrella clothesline". About 1.5 years ago we fell hard in the rankings. Since then "umbrella clothesline" has moved back to the first page, but "clothes drying rack" is stuck on the 3rd page and always with the result being the generic homepage instead of the good sub-page (which used to rank on the first page) that really shows-n-tells about our drying rack. Here are the three pages I am talking about. Home page = http://www.bestdryingrack.com/ Drying rack page = http://www.bestdryingrack.com/clothes-drying-rack-main.html and umbrella clothesline page = http://www.bestdryingrack.com/umbrella-clotheslines.html Any ideas on how to get the drying rack page to start ranking well again? (hopefully better than the generic homepage ranks) A little technical background: the Moz campaign on this site says that the home page has a PA = 42 with 190 LRD's and 344 external links. Both the umbrella clothesline page and the clothes drying rack page have almost equal statistics of PA = 35 with 20 LRD's and 23 external links. My anchor text distribution is maybe unbalanced. The drying rack page has 15 external links with the anchor of "Clothes Drying Rack". But the umbrella clothesline page has 14 external links with the anchor of "outdoor umbrella clothesline" and it ranks on the first page for that search. I can't figure out how to get OSE to tell me anchor text stats for just the homepage and not the whole site since www.bestdryingrack.com/index.html 301's to the plain www.bestdryingrack.com (if you know how, please share) What's wrong with my poor neglected clothes drying rack page? The only way I can get it to show up on the first page is to do a real specific search like "round wooden clothes drying rack" Your help could save a faltering family business. Thank you!
Technical SEO | | GregB1230 -
HTTP Status showing up in opensiteexplorer top pages as blocked by robot.txt file
I am trying to find an answer to this question it has alot of url on this page with no data when i go into the data source and search for noindex or robot.txt but the site is visible in the search engines ?
Technical SEO | | ReSEOlve0 -
HTTP 500 Internal Server Error, Need help
Hi, For a few days know google crawlers have been getting 500 errors from our dedicated server whenever they try to crawl the site. Using the "Fetch as Google" tool under health in webmaster tools, I get "Unreachable page" every time I fetch the homepage. Here is exactly what the google crawler is getting: <code>HTTP/1.1 500 Internal Server Error Date: Fri, 21 Jun 2013 19:52:27 GMT Server: Apache/2.2.15 (CentOS) X-Powered-By: PHP/5.3.3 X-Pingback: [http://www.communityadvocate.com/xmlrpc.php](http://www.communityadvocate.com/xmlrpc.php) Connection: close Transfer-Encoding: chunked Content-Type: text/html; charset=UTF-8 http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> My url is [http://www.communityadvocate.com](http://www.communityadvocate.com/)</code> and here's the screenshot from Goolge webmater http://screencast.com/t/FoWvqRRtmoEQ How can i fix that? Thank you
Technical SEO | | Vmezoz0 -
Can you 301 redirect a page to an already existing/old page ?
If you delete a page (say a sub department/category page on an ecommerce store) should you 301 redirect its url to the nearest equivalent page still on the site or just delete and forget about it ? Generally should you try and 301 redirect any old pages your deleting if you can find suitable page with similar content to redirect to. Wont G consider it weird if you say a page has moved permenantly to such and such an address if that page/address existed before ? I presume its fine since say in the scenario of consolidating departments on your store you want to redirect the department page your going to delete to the existing pages/department you are consolidating old departments products into ?
Technical SEO | | Dan-Lawrence0 -
What to do when you want the category page and landing page to be the same thing?
I'm working on structuring some of my content better and I have a dilemma. I'm using wordpress and I have a main category called "Therapy." Under therapy I want to have a few sub categories such as "physical therapy" "speech therapy" "occupational therapy" to separate the content. The url would end up being mysite/speech-therapy. However, those are also phrases I want to create a landing page for. So I'd like to have a page like mysite.com/speech-therapy that I could optimize and help people looking for those terms find some of the most helpful content on our site for those certain words. I know I can't have 2 urls that are the same, but I'm hoping someone can give me some feedback on the best way to about this. Thanks.
Technical SEO | | NoahsDad0 -
Removing pages from website
Hello all, I am fairly new to the SEOmoz community. But i am working for a company which organizes exhibitons, events and training in Holland. A lot of these events are only given ones ore twice and then we do not organise them any more because they are no longer relevant. Every event has its own few webpages which provide information about the event and are being indexed by Google. In the past we did not remove any of these events. I was looking in the CMS and saw a lot of events of 2008 and older which are being indexed. To clean the website and the CMS i am thinking of removing these pages of old events. The risk is that these pages have some links to them and are getting some traffic, so if i remove them there is a risk of losing traffic and rankings. What would be the wise thing to do? Make a folder with archive or something? Regards, Ruud
Technical SEO | | RuudHeijnen0 -
Is this 404 page indexed?
I have a URL that when searched for shows up in the Google index as the first result but does not have any title or description attached to it. When you click on the link it goes to a 404 page. Is it simply that Google is removing it from the index and is in some sort of transitional phase or could there be another reason.
Technical SEO | | bfinternet0