Ecommerce SEO - Indexed product pages are returning 404's due to product database removal. HELP!
-
Hi all,
I recently took over an e-commerce start-up project from one of my co-workers (who left the job last week). This previous project manager had uploaded ~2000 products without setting up a robot.txt file, and as a result, all of the product pages were indexed by Google (verified via Google Webmaster Tool).
The problem came about when he deleted the entire product database from our hosting service, godaddy and performed a fresh install of Prestashop on our hosting plan. All of the created product pages are now gone, and I'm left with ~2000 broken URL's returning 404's. Currently, the site does not have any products uploaded. From my knowledge, I have to either:
- canonicalize the broken URL's to the new corresponding product pages,
or
- request Google to remove the broken URL's (I believe this is only a temporary solution, for Google honors URL removal request for 90 days)
What is the best way to approach this situation? If I setup a canonicalization, would I have to recreate the deleted pages (to match the URL address) and have those pages redirect to the new product pages (canonicalization)?
Alex
-
Everett,
You're right on the money. I don't think you could have summarized my problem any better. I will take Dana's and your advice and let them sit "indexed" for a while and serve a 404. According to GWT's Index Status, the product pages were indexed about a month ago, so I guess it won't hurt to wait a few more weeks until those pages dropped out of Google's index naturally, especially since the site development won't be done for another 6~7 weeks.
Thanks a bunch for all of your insights
-
Right on Everett. I agree 100%
-
I want to make sure everyone, including myself, understands you Alex. Correct me if I'm wrong, but you're saying that the website is totally new (a start-up) and nothing (at least nothing owned by the company you're with) has ever been on that domain name. While building the site the previous guy accidentally allowed the development version of the site to be indexed, and/or allowed product pages that you don't want on the site at all to be indexed. Since it is a brand new site those "old" pages that were deleted didn't have any external links, and didn't have any traffic from Google or elsewhere outside of the company.
IF that is the case, then you can probably just let those pages stay as 404s. Eventually, since nobody is linking to them, they will drop out of the index on their own.
I wouldn't use the URL removal tool in this case. For one thing, it is a dangerous tool and if you don't have experience with this sort of thing it could do more harm than good. It should only take a few weeks for those URLs that were briefly live and indexed to go away if you are serving a 404 or 410 http header response code on those URLs.
I hope this helps. Please let us know if we have misinterpreted your problem.
-
Understood Alex. Yes, of course you would have to rebuild the pages first before you can 301, but it sounds like you are planning on rebuilding them (otherwise you wouldn't be able to use canonical tags either, because there wouldn't be a page to put them on).
I wouldn't just give up and ask Google to remove all of the old URLs. I agree with what Mike has to say about that below. A 302 is a good option if you are worried about the 404s sitting in the index while you are rebuilding your product pages. If you are still on the same platform (it sounds like that didn't change), I would suggest rebuilding as many of the old URLs as you can (if they were good SEO-friendly URLs). That way you could bypass the 301 redirect. If you want to create your pages so that product options are rolled in and separate colors of things no longer need separate pages, you can then choose whether to 301 redirect those old URLs or simply let them 404.
404s aren't necessarily always a bad thing. Regarding the 2,000 of them you have now, if some of those pages just need to go away, you can let them 404 and they will eventually drop out of Google's index. You aren't required to manually submit them via GWT in order for them to be removed.
-
Hi Mike,
Thanks for weighing in. Recreating all of the old pages seems like a pain in the butt... Besides, the site never launched, so I had no traffic at all. Considering there was no traffic at all to these pages, do you think it's a good idea to go through the URL removal from GWT and purge the broken links completely from Google's index?
- Alex
-
Hi Dana,
Thank you for your advice. I'm new at SEO, so I may be wrong but...
Mapping out the old/new URLs on a spreadsheet and setting up a 301 redirect to the new URLs is not a plausible option in my opinion, mainly because the new URLs literally do not exist (I have not created ANY product pages). According to your suggestion, I would have to create new product pages and do a 301 redirect from the broekn URLs to the newly created pages? Not quite sure if I'm understanding you correctly...
In addition, the previous project manager wasn't SEO-savvy (l'm not either... sigh..), so he didn't know that creating separate pages for a product with multiple attributes (such as flavor and size) would result in major duplicate content issues.
The site is going through some major design/layout overhaul, and I intend to come up with a SEO strategy before creating any categories or products.
Thus...
Do you think it's better to submit a URL removal request on GWT and get rid of the indexed URL's completely? I just re-read Google's policy on URL removal, and it states that as long as I have a 4xx (404 or 410, I'm assuming..) returned for the URLs, Google will honor the removal request.
- Alex
-
Rel Canonical is not quite the right thing for this sort of issue.
If you're worried about the 404s sitting around too long and losing traffic for the moment, you can 302 everything to a landing page, category page, or homepage while you work on setting everything else up. You have two choices at this point.... 1) recreate all of the old pages and old URLs then remove the 302s, or 2) Add new products and new URLs, then as Dana said you'll need to map out all your new product URLs and old URLs to determine what old URL should be 301 redirected where. Then set up your necessary 301s and test that they all work.
-
Hi Alex, I am sorry to hear about this. What a mess, no? If it were me, I wouldn't rely solely on the canonical tag. I would also create a spreadsheet and map all the old URLs to the new URLs and set up 301 redirects from the old to the new. 2,000 isn't too bad. You can probably knock them out in 2-3 days...but be sure to test all of the 301s and make sure they are performing the way you expect them to. Hope that helps a little!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My url disappeared from Google but Search Console shows indexed. This url has been indexed for more than a year. Please help!
Super weird problem that I can't solve for last 5 hours. One of my urls: https://www.dcacar.com/lax-car-service.html Has been indexed for more than a year and also has an AMP version, few hours ago I realized that it had disappeared from serps. We were ranking on page 1 for several key terms. When I perform a search "site:dcacar.com " the url is no where to be found on all 5 pages. But when I check my Google Console it shows as indexed I requested to index again but nothing changed. All other 50 or so urls are not effected at all, this is the only url that has gone missing can someone solve this mystery for me please. Thanks a lot in advance.
Intermediate & Advanced SEO | | Davit19850 -
Can a duplicate page referencing the original page on another domain in another country using the 'canonical link' still get indexed locally?
Hi I wonder if anyone could help me on a canonical link query/indexing issue. I have given an overview, intended solution and question below. Any advice on this query will be much appreciated. Overview: I have a client who has a .com domain that includes blog content intended for the US market using the correct lang tags. The client also has a .co.uk site without a blog but looking at creating one. As the target keywords and content are relevant across both UK and US markets and not to duplicate work the client has asked would it be worthwhile centralising the blog or provide any other efficient blog site structure recommendations. Suggested solution: As the domain authority (DA) on the .com/.co.uk sites are in the 60+ it would risky moving domains/subdomain at this stage and would be a waste not to utilise the DAs that have built up on both sites. I have suggested they keep both sites and share the same content between them using a content curated WP plugin and using the 'canonical link' to reference the original source (US or UK) - so not to get duplicate content issues. My question: Let's say I'm a potential customer in the UK and i'm searching using a keyword phrase that the content that answers my query is on both the UK and US site although the US content is the original source.
Intermediate & Advanced SEO | | JonRayner
Will the US or UK version blog appear in UK SERPs? My gut is the UK blog will as Google will try and serve me the most appropriate version of the content and as I'm in the UK it will be this version, even though I have identified the US source using the canonical link?2 -
SEO for Product Pages Deal that will last One Day Only
For an Ecommerce website I am required to create two pages. 1) One that will be displaying the "Deal of the day", which is basically a summary of the product on sale and another 2) product page where the actual product-deal resides. "Deal of the day" page Fixed url e.g. homepage.com/deal-of-the-day Product description summary Go to product-deal & Buy Now Button Content changes everyday Product Deal Page Similar to other products, sometimes will be a group of products, coupons etc. Product deals will be stored for later re-use Not visible from the main product catalogue These products are most of the time the same products from the catalogue but different copy Recommendations? Thanks!
Intermediate & Advanced SEO | | raulreyes0 -
Pagination on a product page with reviews spread out on multiple pages
Our current product pages markup only have the canonical URL on the first page (each page loads more user reviews). Since we don't want to increase load times, we don't currently have a canonical view all product page. Do we need to mark up each subsequent page with its own canonical URL? My understanding was that canonical and rel next prev tags are independent of each other. So that if we mark up the middle pages with a paginated URL, e.g: Product page #1http://www.example.co.uk/Product.aspx?p=2692"/>http://www.example.co.uk/Product.aspx?p=2692&pageid=2" />**Product page #2 **http://www.example.co.uk/Product.aspx?p=2692&pageid=2"/>http://www.example.co.uk/Product.aspx?p=2692" />http://www.example.co.uk/Product.aspx?p=2692&pageid=3" />Would mean that each canonical page would suggest to google another piece of unique content, which this obviously isn't. Is the PREV NEXT able to "override" the canonical and explain to Googlebot that its part of a series? Wouldn't the canonical then be redundant?Thanks
Intermediate & Advanced SEO | | Don340 -
Putting "noindex" on a page that's in an iframe... what will that mean for the parent page?
If I've got a page that is being called in an iframe, on my homepage, and I don't want that called page to be indexed.... so I put a noindex tag on the called page (but not on the homepage) what might that mean for the homepage? Nothing? Will Google, Bing, Yahoo, or anyone else, potentially see that as a noindex tag on my homepage?
Intermediate & Advanced SEO | | Philip-DiPatrizio0 -
Do 404 Errors hurt SEO ?
I recently did a crawl test and it listed over 10,000 pages, but around 282 of them generated 404 errors (bad links) I'm wondering how much this hurts overall SEO and if its something I should focus on fixing asap ? Thanks.
Intermediate & Advanced SEO | | RagingBull0 -
Page Titles SEO Title
Hi, I run an e-commerce store and within the CMS I define the SEO title, SEO description and SEO keywords for each item. I spoke to a SEO firm who advised me to start every product title with the colour, as this will reduce the duplicate page titles and serve me well in the future. Whats everyones view on this? Does naming something Grey Armani Jeans | Armani Jeans from Designer Boutique stand up better against Armani Jeans Grey | Armani Jeans from Designer Boutique? Any help or tips on how to format the page titles and descriptions would be great. Thanks Will
Intermediate & Advanced SEO | | WillBlackburn0 -
Best solution to get mass URl's out the SE's index
Hi, I've got an issue where our web developers have made a mistake on our website by messing up some URL's . Because our site works dynamically IE the URL's generated on a page are relevant to the current URL it ment the problem URL linked out to more problem URL's - effectively replicating an entire website directory under problem URL's - this has caused tens of thousands of URL's in SE's indexes which shouldn't be there. So say for example the problem URL's are like www.mysite.com/incorrect-directory/folder1/page1/ It seems I can correct this by doing the following: 1/. Use Robots.txt to disallow access to /incorrect-directory/* 2/. 301 the urls like this:
Intermediate & Advanced SEO | | James77
www.mysite.com/incorrect-directory/folder1/page1/
301 to:
www.mysite.com/correct-directory/folder1/page1/ 3/. 301 URL's to the root correct directory like this:
www.mysite.com/incorrect-directory/folder1/page1/
www.mysite.com/incorrect-directory/folder1/page2/
www.mysite.com/incorrect-directory/folder2/ 301 to:
www.mysite.com/correct-directory/ Which method do you think is the best solution? - I doubt there is any link juice benifit from 301'ing URL's as there shouldn't be any external links pointing to the wrong URL's.0