How should I go about repairing 400,000 404 error pages?
-
My thinking is to make a list of most linked to and most trafficked error pages, and just redirect those, but I don't know how to get all that data because i can't even download all the error pages from Webmaster Tools, and even then, how would i get backlink data except by checking each link manually?
Are there any detailed step-by-step instructions on this that I missed in my Googling?
Thanks for reading!!
-
Try using ScreamingFrog (app for mac and pc). It's free and will crawl your site and list all the errors an dresponse codes 9inc 404s) you can export these as a CSV file.
HTH
Steve
-
Thanks for responding!
The site wasn't deleted. I will have to investigate what caused all these 404s. I suspect a redesign.
Is there a way get a full list of 404 pages with backlink information if possible for each URL?
Thanks!
-
I'm guessing that you deleted a whole site without 301'ing.
What is the base URL? (myDomain.com)?
There is a chance that you could use an .HTACCESS file with some redirect rules to send whole batches to a new domain but these are all guesses without further detail from you.
Good luck...
Steve
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Doorway page penalty
Has Google changed their interpretation of Doorway pages?We do not sell widgets but allow me to use Widget for this example;If we sold 25 very different widgets an online vendor would typically have 1 "mother" website with 25 different inner pages, each page to explain each type of widget they sell.However, for the past 9 years our approach is to have 25 different websites, one for each widget. With these 25 sites we concentrated on ranking the home page only . All these sites link back to our (No idexed) "Mother' site via no follow links where we have our Shopping Cart and Terms of Business. We did this partly to avoid having 25 separate Shopping Carts and to avoid having to change our Terms 25 times each time that became necessary. But yes we also did this as it was so much easier to rank each different type of widget in the SERPS. Also we think its a better user experience as in our business buyers of yellow widgets will not be interested in blue widgetsWe have been reading for years that google does not like doorways pages but we were not 100% certain if they might regard our sites as such .This is because our approach has worked great for nine years. That is until December last year when all 95% our sites fell dramatically in the SERPS usually from page 1 to page 2 or 3. First thing we did was to go through all our sites and search for the obvious; toxic links, duplicate content, keyword density, https issues, mobility issues, anchor text, etc etc and of course content. We found no obvious problems that could affect 95% of the sites at the same time but we ordered new homepage content for most of our sites from expert seo writers. However, after putting on this new content 3 -4 weeks ago our sites have not moved up the SERPS at all.So we are left with the inescapable conclusion that our problem is because google sees and devalues our sites as doorway pages especially as 95% of your sites have been affected all at the same time Would any SEO experts on this forum agree or be able to offer an opinion?If so, what might be the solution going forward? We have 2 solutions under consideration;1) Remove all links from each of our 25 sites to our "mother Site" and put a shopping cart and our TOS on each of the 25 sites so they are all truly independent stand alone websites.2) Create 25 inner pages on our mother site (after removing the no index) , for each of the 25 widgets we sell , then 301 each of the 25 individual sites home pages to its inner page on the mother site . I think this might be the best solution partly as almost all of our higher ranking competitors are ranking their inner pages not their homepage. But I worry if these 25 sites will really pass much link juice if they have been devalued by Google.?Any advice will be gratefully received.
Intermediate & Advanced SEO | | apcsilver90 -
How do we decide which pages to index/de-index? Help for a 250k page site
At Siftery (siftery.com) we have about 250k pages, most of them reflected in our sitemap. Though after submitting a sitemap we started seeing an increase in the number of pages Google indexed, in the past few weeks progress has slowed to a crawl at about 80k pages, and in fact has been coming down very marginally. Due to the nature of the site, a lot of the pages on the site likely look very similar to search engines. We've also broken down our sitemap into an index, so we know that most of the indexation problems are coming from a particular type of page (company profiles). Given these facts below, what do you recommend we do? Should we de-index all of the pages that are not being picked up by the Google index (and are therefore likely seen as low quality)? There seems to be a school of thought that de-indexing "thin" pages improves the ranking potential of the indexed pages. We have plans for enriching and differentiating the pages that are being picked up as thin (Moz itself picks them up as 'duplicate' pages even though they're not. Thanks for sharing your thoughts and experiences!
Intermediate & Advanced SEO | | ggiaco-siftery0 -
Redirecting homepage to internal page (2nd Tier page)
We are planning to experiment redirecting our homepage to one of the 2nd tier page. I mean....example.com to example.com/page. We need this page to rank well, but it doesn't have much internal links or external back-links, so we opt for this redirect. Advantage with this page is, it has "keyword" we want to rank for in URL. "page" in example.com/page. Will this help or hurt us in SEO? I think we are missing keyword in our root domain, so interested to highlight this page. Thanks, Satish
Intermediate & Advanced SEO | | vtmoz0 -
Outside Top 10 Even though - Higher Domain/Page Authority/Higher On Page Grade
Hi, Note: this is for Australian search results - for people in Perth.
Intermediate & Advanced SEO | | HeadStud
The website is: http://thedj.com.au I am trying to optimise for the keyword 'perth wedding dj', but also 'wedding dj perth' and for some reason my website isn't even in the top 10 results. Here is what's weird though: My on-page grade with the On-Page Grader for the keyword 'wedding DJ perth' is an 'A' for http://thedj.com.au (http://awesomescreenshot.com/0135135hca) When checking the Keyword Difficulty in the Google Australia search enginge for 'wedding DJ perth' - there are 4 results which have a lower domain authority than 15 (in fact one result has a domain authority of 1) - http://awesomescreenshot.com/03f5134zd1 http://thedj.com.au has a Domain Authority of 23/100 and a Page Authority of 34/100. (http://awesomescreenshot.com/0bb5134tb8) So seeing as the page has gotten an A for on-page optimisation for the keyword 'wedding DJ Perth' and has a higher domain authority then many results in the top 10... why isn't it in the Top 10?! Bonus Question:
Why is DJ Avi showing up at the top of search results (Local listing) depsite the fact that:
a) He has no website to link to
b) No reviews for his listing
c) No keywords that I can see (other than the fact that he's a DJ)
Screenshot: http://awesomescreenshot.com/05151349cb Meanwhile our Local Places - Thanks,
Kosta
http://www.headstudios.com.au0 -
Dynamic pages
Hello Team, How can we create dynamic pages or more pages on website but maintaining SEO standards.
Intermediate & Advanced SEO | | Obbserv0 -
Need help with huge spike in duplicate content and page title errors.
Hi Mozzers, I come asking for help. I've had a client who's reported a staggering increase in errors of over 18,000! The errors include duplicate content and page titles. I think I've found the culprit and it's the News & Events calender on the following page: http://www.newmanshs.wa.edu.au/news-events/events/07-2013 Essentially each day of the week is an individual link, and events stretching over a few days get reported as duplicate content. Do you have any ideas how to fix this issue? Any help is much appreciated. Cheers
Intermediate & Advanced SEO | | bamcreative0 -
Does a page on a site with high domain authority build page authority easier? i.e. less inbound links?
Is this also why people build backlinks to their BBB profiles, Yellowpages Profiles, etc. i.e. why do people build backlinks to other pages that link to them? Wouldn't it be more beneficial to just build that backlink directly to your target?
Intermediate & Advanced SEO | | adriandg0 -
Will Google Visit Non-Canonicalized Page Again and Return Its Page's Original Ranking?
I have 2 questions about canonicalization. 1. Will Google ever visit Page A again if after it has been canonicalized to Page B? 2. If Google will still visit Page A and found that it is not canonicalizing to Page B already, will the original rankings and traffic of Page A returned to the way before it's canonicalized? Thanks.
Intermediate & Advanced SEO | | globalsources.com0
