Page loads fine for users but returns a 404 for Google & Moz
-
I have an e-commerce website that is built using Wordpress and the WP E-commerce plug-in, the products have always worked fine and the pages when you view them in a browser work fine and people can purchase the products with no problems.
However in the Google merchant feed and in the Moz crawl diagnostics certain product pages are returning a 404 error message and I can't work out why, especially as the pages load fine in the browser.
I had a look at the page headers and can see when the page does load the initial request does return a 404 error message, then every other request goes through and loads fine. Can anyone help me as to why this is happening?
A link to the product I have been using to test is: http://earthkindoriginals.co.uk/organic-clothing/lounge-wear/organic-tunic-top/
Here is a part of the header dump that I did:
http://earthkindoriginals.co.uk/organic-clothing/lounge-wear/organic-tunic-top/
GET /organic-clothing/lounge-wear/organic-tunic-top/ HTTP/1.1
Host: earthkindoriginals.co.uk
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: __utma=159840937.1804930013.1369831087.1373619597.1373622660.4; __utmz=159840937.1369831087.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); wp-settings-1=imgsize%3Dmedium%26hidetb%3D1%26editor%3Dhtml%26urlbutton%3Dnone%26mfold%3Do%26align%3Dcenter%26ed_size%3D160%26libraryContent%3Dbrowse; wp-settings-time-1=1370438004; __utmb=159840937.3.10.1373622660; PHPSESSID=e6f3b379d54c1471a8c662bf52c24543; __utmc=159840937
Connection: keep-alive
HTTP/1.1 404 Not Found
Date: Fri, 12 Jul 2013 09:58:33 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
X-Pingback: http://earthkindoriginals.co.uk/xmlrpc.php
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 6653
Connection: close
Content-Type: text/html; charset=UTF-8 -
Thanks for the help guys, it is good to actually have a direction to look in now, I was just completely stuck before. I will post any updates I have.
-
Hello,
The status returned is 404 not found, this is independent of whether the page is loaded or not.
There is something that is generating that code either htaccess, some php code, maybe some redirection, a misconfigured rewrite, look for what can be, with that code, pages are not indexed.
Sorry for my english.
Best regards,
Carlos
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Issue with AMP pages
Hello, We have implemented AMP on our blog pages, but now some of the Web pages are also being shown like AMP pages. ( no footer and no navigation ) What could have gone wrong ? Thanks
Intermediate & Advanced SEO | | Johnroger0 -
Google serving wrong page...
Hi, When you Google: "Los Angeles divorce attorney", you will see this site on the 5th page of the SERPS: www.berenjifamilylaw.com/blog/. For some reason, Google is serving the BLOG page as opposed to the homepage. This has been going on now for several weeks. Any tips on how to fix this? Obviously, the Homepage is more relevant and has more links going to it, so not sure why it's happening. Would you just leave it alone? Would you use robots.txt to block Google from crawling the BLOG post page? Thanks.
Intermediate & Advanced SEO | | mrodriguez14400 -
Taking Back Google+ Page Managed By Advertiser
The Google+ Page that appears under local listings in Google for a company I work for, was setup by an advertiser, such as Yellow Pages. I'm not sure how they got control of it, but apparently somebody at this agency set it up, and verified the url. The G+ page is now outdated, and needs to be updated. More importantly, I have to manage the page, and not this third party. I sent a request to Google to transfer ownership. My question is, has anybody had any luck in getting control of a Google+ page which was setup by the wrong party, back? As a backup, I've initiated a plan of creating a new Google+ page with the account info from the WMT's account of the url. I figure, that might have more authority than a page erroneously setup by a third party. My hope is it would eventually replace the Google+ page the currently shows in local returns. Any thoughts or comment would be appreciated.
Intermediate & Advanced SEO | | alrockn0 -
How should I go about repairing 400,000 404 error pages?
My thinking is to make a list of most linked to and most trafficked error pages, and just redirect those, but I don't know how to get all that data because i can't even download all the error pages from Webmaster Tools, and even then, how would i get backlink data except by checking each link manually? Are there any detailed step-by-step instructions on this that I missed in my Googling? Thanks for reading!!
Intermediate & Advanced SEO | | DA20130 -
Do you add 404 page into robot file or just add no index tag?
Hi, got different opinion on this so i wanted to double check with your comment is. We've got /404.html page and I was wondering if you would add this page to robot text so it wouldn't be indexed or would you just add no index tag? What would be the best approach? Thanks!
Intermediate & Advanced SEO | | Rubix0 -
Google cached pages and search terms
Here's something I noticed. We have a rank A page and it's ranking 10 on Google search results. When I hover my mouse over our search result, Google gives us a preview, but Google also highlights in red where the search keyword is present on the page. Reviewing our page, even though we have it as the h1 header and intro paragraph, Google is highlighting it half way down the page. Any ideas why? I review rank 1 - 5 and Google highlights the keyword on the intro paragraph and h1 header Have you guys experienced anything like this? It makes me think..Google could be crawling my site and thinking I haven't got it in the h1 or intro paragraph etc.. Thoughts?
Intermediate & Advanced SEO | | Bio-RadAbs0 -
2 page titles, 1 url in Google SERPS: WTF!?!?
Hey guys, Hope everybody is having a good day. Today i came across something i have never seen in the serps before that i would like to share and getting feedback on. When i search for 'woonverzekering' on google.nl #1 is: **Url: ** www.independer.nl/woonverzekering/intro.aspx
Intermediate & Advanced SEO | | PrizeWize
**page titel: **Woonverzekering - Independer.nl When i search for 'woonhuisverzekering' on google.nl #1 is: **Url: ** www.independer.nl/woonverzekering/intro.aspx
page titel: Woonhuisverzekering? Vergelijk alle soorten woonverzekeringen - Independer.nl So basically 2 different queries show the same url with 2 different page titles in the serps. The only 'weird' thing i could find was a nobreakspace in the page title code: Woonhuisverzekering? Vergelijk alle soorten woonverzekeringen - Independer.nl I'm i missing something completely obvious here? Is this a commonly used technique. Is the page title getting chopped up because of ? What are they doing to get 2 page title results on 1 url?0 -
Google Page Rank Dead?
Does PR still work? I have sites that have PR3 and get almost no traffic and sites that are PR1 and get thousands of uniques per month. My PR on my main sites haven't moved for about 7 years, even though we've grown significantly. I know lots of you are going to jump in with get the MOZ toolbar, which I already have done, and I agree, it's great ... But can anyone tell me about what's going on with Google PR? Is it still active? Or has Google abandoned? I noticed that the Google toolbar is not even available for Google Chrome. That should say something ... If you like this question, do me a favor, and give me a THUMBS UP!
Intermediate & Advanced SEO | | applesofgold2