Intermittent 404 - What causes them and how to fix?
-
Hi!
I'm working on a client site at the moment and I've discovered a couple of pages that are 404ing but producing a 200 OK response.
However, I have checked these URLs again and some are now producing a 404 Error response. No changes have been made (that I'm aware of) so it appears that the URLs are returning both 200 OK and 404 Error responses intermittently.
Any ideas what could cause this and the best solution?
Thanks!
-
Hi Daniel,
That does seem very odd!
There can be various different things at play here in my experience:
- Publishing and un-publishing in the CMS - Sometimes clients and CMS users will switch the page publish status around which can make it difficult to keep track of whether the page exists or not
- Removal of redirects - Much the same as above, there's the possibility that someone is adding/removing redirects. This could also be something to do with the removal of the server level redirects file, be that htaccess or IIS.
- Crawling tool issue - If you're using a crawling tool (especially if doing so with JS rendering active), sometimes this can put a lot of pressure on the server and it can spit out the occasional error where there actually isn't one
It could also be a problem with how you're handing hard 404 errors vs soft 404s - i.e. actual not founds vs pages that don't function but the server is under the impression that they're fine.
Best of luck!
Sean
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Blogs Not Getting Indexed Intermittently - Why?
Over the past 5 months many of our clients are having indexing issues for their blog posts.
Technical SEO | | JohnBracamontes
A blog from 5 months ago could be indexed, and a blog from 1 month ago could be indexed but blogs from 4, 3 and 2 months ago aren't indexed. It isn't consistent and there is not commonality across all of these clients that would point to why this is happening. We've checked sitemap, robots, canonical issues, internal linking, combed through Search Console, run Moz reports, run SEM Rush reports (sorry Moz), but can't find anything. We are now manually submitting URLs to be indexed to try and ensure they get into the index. Search console reports for many of the URLs will show that the blog has been fetched and crawled, but not indexed (with no errors). In some cases we find that the blog paginated pages (i.e. blog/page/2 , blog/page/3 , etc.) are getting indexed but not the blogs themselves. There aren't any nofollow tags on the links going to the blogs either. Any ideas? *I've added a screenshot of one of the URL inspection reports from Search Console alt text0 -
Location-Aware Browsing issue. Could it cause a drop in rankings
Hello Mozzers. A client has seen big drop in their organic rankings in google.co.uk. Desktop results have dropped from page one to page three and mobile results have completely gone. They have recently added Location-Aware Browsing to the site so that it attempts to find you location and add this to the search box. However this appears to be in a loop and doesn't then turn off once you have allowed it. It keeps appearing in both Firefox and Chrome and asking for permission to use you location. Could this have resulted in the ranking drop? There have also rolled out a new mobile version and this has the same issue. Any ideas? Thanks in advance TH
Technical SEO | | highwayfive0 -
How to fix duplicate content caused by tags?
I use SEMRush, and the issue they are finding is I have 30 duplicate content issues. All seem to be caused by the tags I add in my portfolio pieces. I have looked at my SEO settings (taxonomies, etc) in the Wordpress site, and don't know what I am doing wrong....any advice how to fix? I have attached a screen shot VsYv2wY
Technical SEO | | cschwartzel0 -
Instead of a 301, my client uses a 302 to custom 404
I've found about 900 instances of decommissioned pages being redirected via 302 to a 404 custom page, even when there's a comparable page elsewhere on the site or on a new subdomain. My recommendation would be to always do a 301 from the legacy page to the new page, but since they're are so many instances of this 302->404 it seems to be standard operating procedure by the dev team. Given that at least one of these pages has links coming from 48 root domains, wouldn't it obviously be much better to 301 redirect it to pass along that equity? I don't get why the developers are doing this, and I have to build a strong case about what they're losing with this 302->404 protocol. I'd love to hear your thoughts on WHY the dev team has settled on this solution, in addition to what suffers as a result. I think I know, but would love some more expert input.
Technical SEO | | Jen_Floyd0 -
How to fix errors and warnings on a wordpress.com hosted site ?
Hello Mozers, I've 18 4xx errors ,812 duplicate page content and 412 duplicate page titles with about 605 too many links warning and about 4900 notices.. My website is hosted on wordpress.com and I just do not understand how do i fix these errors . To add on, last week the errors were lesser by 150 !! How do I get these issues fixed ? Please assist !!! Thanks , VIkash
Technical SEO | | mysayindia0 -
Will rel=canonical cause a page to be indexed?
Say I have 2 pages with duplicate content: One of them is: http://www.originalsite.com/originalpage This page is the one I want to be indexed on google (domain rank already built, etc.) http://www.originalpage.com is more of an ease of use domain, primarily for printed material. If both of these sites are identical, will rel=canonical pointing to "http://www.originalsite.com/originalpage" cause it to be indexed? I do not plan on having any links on my site going to "http://www.originalsite.com/originalpage", they would instead go to "http://www.originalpage.com".
Technical SEO | | jgower0 -
404 handling the right way
Hi, Currently when a page is not found I make a 301 redirect to a 404 page should I really do a redirect or maybe a rewrite of the 404 page without redirection? Thanks, Asaf
Technical SEO | | AsafY0 -
Slash at end of URL causing Google crawler problems
Hello, We are having some problems with a few of our pages being crawled by Google and it looks like the slash at the end of the URL is causing the problem. Would appreciate any pointers on this. We have a redirect in place that redirects the "no slash" URL to the "slash" URL for all pages. The obvious solution would be to try turning this off, however, we're unable to figure our where this redirect is coming from. There doesn't appear to be an instruction in our .htaccess file doing this, and we've also tried using "DirectorySlash Off" in the .htaccess file, but that doesn't work either. (if it makes a difference it is a 302 redirect doing this, not a 301) If we can't get the above to work, then the other solution would be to somehow reconfigure the page so that it is recognizable with the slash at the end by Google. However, we're not sure how this would be done. I think the quickest solution would be to turn off the "add slash" redirect. Any ideas on where this command might be hiding, and how to turn it off would be greatly appreciated. Or any tips from people who have had similar crawl problems with google and any workarounds would be great! Thanks!
Technical SEO | | onetwentysix0