Is the If-Modified-Since HTTP Header still relevant?
-
I'm relatively new to the technical side of SEO and have been trying to brush up my skills by going through Google's online Web-master Academy, which suggests that you need a If-Modified-Since HTTP Header tag on your site. I checked and apparently our web server doesn't support this.
I've been told by a good colleague that the If-Modified-Since tag is no longer relevant as the spiders will frequently revisit a site as long as you regularly update and refresh the content (which we do).
However our site doesn't seem to of been reindexed for a while as the cached version's are still showing the pages from over a month ago.
So two question really - is the If-Modified-Since HTTP Header still relevant and should I make sure this is included?
And is there anything else I should be doing to make sure the spiders crawl our pages? (apart from keeping them nice, fresh and useful)
-
If the webserver does not support (or the admin does not want to enable) this feature you could always have your frontend-templates have a small string wich holds the date/time when the page was last updated. Something along the lines "last updated on: ...." at the bottom or top of the content area. It's also an useful bit of information for users.
-
Hi Annie
I'm surprised there hasn't been lots of answers to your question.
Check-out this video here on SEOmoz entitled "Whiteboard Interview - Google's Matt Cutts on Redirects, Trust + More" featuring Matt Cutts being asked some questions by Rand. It opens with a partial answer to your first question:
"These days we use it a little less" (2 years ago) ~ basically means that in locations such as the US, most of Europe, Japan... & so on, where Bandwidth is rarely an issue anymore, 'If-Modified-Since' isn't taken notice of, it's not worth including anymore.
In say developing countries where bandwidth is sometimes still on the low side, it may still be used, hence why a sweeping 'it doesn't matter anymore' statement wasn't given.
**Your second question: **
- Content, fresh unique value-adding content that is, that's engaging and shareable, is always a positive aspect to work on, which in turn can lead to some awesome new links. This encourages the bots to visit more regularly.
- Ensuring that your site doesn't have any technical issues (say causing significant downtime).
- Ensuring that Robots.txt isn't wrongly disallowing any pages from being crawled.
- Keeping an eye on Google Webmaster Tools (& Bing Webmaster Tools) for any messages or errors.
- You can alter the crawl rate in GWT, though is usually best to leave it on the default auto setting.
Hope that helps,
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google is still indexing the old domain a year after 301 redirects are put in place
Hi there, You might have experienced this before but for me this is the first. A client of mine moved from domain A (www.domainA.com) to domain B (www.domainB.com). 301 redirects are all in place for over a year. But the old domain is still showing in Google when you search for "site:domainA.com" The HTTP Header check shows this result for the URL https://www.domainA.com/company/cookie-policy.aspx HTTP/1.1 301 Moved Permanently =>
Technical SEO | | iQi
Cache-Control => private
Content-Length => 174
Content-Type => text/html; charset=utf-8
Location => https://www.domain_B_.com/legal/cookie-policy
Server => Microsoft-IIS/10.0
X-AspNetMvc-Version => 5.2
X-AspNet-Version => 4.0.30319
X-Powered-By => ASP.NET
Date => Fri, 15 Mar 2019 12:01:33 GMT
Connection => close Does the redirect look wrong? The change of address request was made on Google Console when the website was moved over a year ago. Edit: Checked the domainA.com on bing and it seems that its not indexed, and replaced with domainB.com, which is the right. Just Google is indexing the old domain! Please let me know your thoughts on why this is happening. Best,0 -
Header Logo and H1
I would like to use the website logo on our site as the H1 for the homepage (see top 'Covers' logo http://www.covers.com/index.aspx ). I have found one of our sister sites with code like this Gracenote with CSS that sets the text to 'transparent' and the background image to the logo. My question is would the text be consumed or ignored by the robot? Unfortunately, wedging an H1 in anywhere else is not possible at this time, so this is my best option. Is this change worthwhile? THanks in advance.
Technical SEO | | evansluke0 -
How to fix HTTP/HTTPS duplicate content
I recently installed an SSL certificate on the site: https://libertywholesalesupply.com Moz is now reading thousands of duplicate content pages because it is reading both http and https. I set up the configuration in Magento to auto-redirect the base URL, created a permanent redirect for the URL in the SEO settings, and adjusted the canonical settings. What am I missing??
Technical SEO | | adamxj20 -
Google showing https:// page in search results but directing to http:// page
We're a bit confused as to why Google shows a secure page https:// URL in the results for some of our pages. This includes our homepage. But when you click through it isn't taking you to the https:// page, just the normal unsecured page. This isn't happening for all of our results, most of our deeper content results are not showing as https://. I thought this might have something to do with Google conducting searches behind secure pages now, but this problem doesn't seem to affect other sites and our competitors. Any ideas as to why this is happening and how we get around it?
Technical SEO | | amiraicaew0 -
Does all in one seo pack still have a rel canonical issue?
Hi All, I know that the all in one had errors in its rel canonical links on Wordpress but I wondered if this has been fixed. I get mixed info on the web. Anyone know for sure? Thanks!
Technical SEO | | xvpn9020 -
Page has Noindex, nofollow, still ranks #1
Hi there, I have a question about a few pages on our site, whom has a no index, nofollow meta tag but they are still indexed and even rank number one in our market for the term. How is that possible or is it that Google just ignores the tags when they think it´s an error from our side? The url is www.drogisterij.net/kilo_killer and the keyword is kilo killer. We rank number 1 if you search from Google.nl Anyone have seen it before and know why this might be? Thanks in advance.
Technical SEO | | JaapWillemDrogisterij0 -
HTTP status code 302
When running Seomoz crawl report, I saw a link site https://www.vietnamvisacorp.com/?req=apply-visa with status code 302 and empty title tag Is this an error? and what should I do to fix this problem? Thank you,
Technical SEO | | JohnHuynh0 -
What's the best way to deal with an entire existing site moving from http to https?
I have a client that just switched their entire site from the standard unsecure (http) to secure (https) because of over-zealous compliance issues for protecting personal information in the health care realm. They currently have the server setup to 302 redirect from the http version of a URL to the https version. My first inclination was to have them simply update that to a 301 and be done with it, but I'd prefer not to have to 301 every URL on the site. I know that putting a rel="canonical" tag on every page that refers to the http version of the URL is a best practice (http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=139394), but should I leave the 302 redirects or update them to 301's. Something seems off to me about the search engines visiting an http page, getting 301 redirected to an https page and then being told by the canonical tag that it's actually the URL they were just 301 redirected from.
Technical SEO | | JasonCooper0