Handling a Huge Amount of Crawl Errors
-
HI all,
I am faced with a crawl errors issue of a huge site (>1MiO pages) for which I am doing On-page Audit.
-
404 Erorrs: >80'000
-
Soft 404 Errors: 300
-
500 Errors: 1600
All of the above reported in GWT.
Many of the error links are simply not present on the pages "linked from". I investigated a sample of pages (and their source) looking for the error links footprints and yet nothing.
What would be the right way to address this issue from SEO perspective, anyway? Clearly. I am not able to investigate the reasons since I am seeing what is generated as HTML and NOT seeing what's behind.
So my question is: Generally, what is the appropriate way of handling this?
-
Telling the client that he has to investigate that (I gave my best to at least report the errors)
-
Engaging my firm further and get a developer from my side to investigate?
Thanks in advance!!
-
-
Usually an on page audit lists all of the problems and possible reasons why they are happening, not in depth info on how to fix all the issues. That is usually the next phase, "do you want me to work on the site or do you want your dev team to track down the cause of the issues and fix them"
It also depends what type of contract you have with him of course.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Moz is showing a canonical error that dont belong.
Hi guys, and thanks for this excellent source of information. i have an issue with the moz system because is telling to me that i dont have canonical instructions but i have canonical instructions on all my pages, so... im confused because maybe im not understanding what the system want to show to me. if you can help me i will be very gratefull. here you can see a page that have the canonical instruction. https://drive.google.com/file/d/14U_-Sgu_NQaB7kMBH3AguHQMHyHX9L8X/view?usp=sharing and here you can see what is reporting to me the MOZ system. https://drive.google.com/file/d/1pqgSC-V9WOyBPvQEr06pbqpLf_w7-q8J/view?usp=sharing this is happening on 19 pages, and all the 19 pages have the canonical instruction.
On-Page Optimization | | b-lab
thanks in advance guys.0 -
HTTPS and HTTP both exist! How to handle?
I was asked to do some SEO work for a website and learned that just 6 weeks ago, their webmaster added an HTTPS instance of the site. Their backlinks all point to HTTP and the 6 pages that are already ranking are all on the HTTP site. I'm afraid to rock the boat by redirecting the site from HTTP to HTTPS as we may lose rank. What are some suggestions? If I just pull down the HTTPS will that hurt us? Would you just go ahead and redirect it? IF so, would you do each page individually or as a whole?
On-Page Optimization | | dk80 -
How do you handle different business locations for search?
Would like to get peoples suggestion on how you handle different business locations in different cities. We have tried multiple tactics for different clients. Some have worked better than others. Example: We have a window company that does great in Austin. Now they want to move into Dallas. In the past, we have created "landing pages" optimized for that locality. However, with Googles new updates I dont think this will work anymore. With others, we have added a "tab" and have similar pages of the entire site optimized for that location. This seems to have been working better. What are your suggestion of how you handle different locations from city to city. Any input is appreciated! Thanks! Charles
On-Page Optimization | | seomozinator0 -
How to handle wordpress tags
Hi fellow SEO mozzers, I am getting 'duplicate content' errors when our site is crawled, mainly down to our WordPress blog and how we have handled tags. Currently, they are being crawled and as such are regarded as duplicate pages. I have read several different articles on how to handle tags. Some suggest noindex the tag URL's. Others suggest to optimize them and allow them to be indexed since Google has confirmed they won't penalize a WordPress site for having archive pages that publish and point to the same content. It will select the best link to represent the cluster of links. Over the past few months, nearly 4% of our WordPress traffic have been referred by tag pages listed in search engines. Initially I was going to noindex the tag pages, but going on the above info I wonder should I leave them as they are? Or is the issue that having duplicate content will lead to inefficient crawling? Any views/opinions on how best to handle this?
On-Page Optimization | | efink0 -
How to handle long dynamic meta tags?
Hi All, I have a site that has upwards of 40 000 pages and I'm redeveloping it so really want to get some SEO elements spot on for the new development. Hoe do I go about handling the following: The user creates a title for their advert which I use as the meta title. The problem is titles are quite often longer that the accepted lengths. How should I handle this? String manipulation down to the desired size, leave it as is or is there another solution? The meta descriptionn is pulled from a summary they created as part of their profile. Is this the right way to do it? Any advice would be appreciated. Ross
On-Page Optimization | | Mulith0 -
Why that many Crawl Diagnostics errors, false?
Hi, my fist question, trying to understand how seomoz pro works. I have about 680 crawl erros, but when I check the details, I found this: 1. many 403 errors ( I think from all links). > I have tested my web site (telcelsolcom.com) in other tools and all says OK 200 response. 2. many title and content duplicates, but the system is showing as duplicated pages with and without the www. > I have a 301 redirect from non www to with www and it is working ok. Do I have false errors? What am I doing wrong? thanks.
On-Page Optimization | | hugoaf0 -
Handling Deleted Pages
The backstory: My site has a fantasy sportsbook where users can place "bets" on pretty much anything. Each game has a unique matchup page, where the odds are displayed and any conversation about the game takes place. Example here. About 95% of the games are automatically graded, but I have to manually grade the rest. Therefore, as soon as every game starts I check to see if any users have made a pick on it, and if not I delete it because it reduces my workload. The problem: About 15% of my search-driven traffic is queries for games that no longer exist, which makes sense because nobody bets on the super obscure games and these games are very easy to rank for. I am currently redirecting them to my 404 page but I'm worried that all of these hits are hurting my reputation with the big G. Would it be better to noindex all of these pages at first and take the noindex away as soon as I'm positive that the game will stay?
On-Page Optimization | | PatrickGriffith0 -
How To Prevent Crawling Shopping Carts, Wishlists, Login Pages
What's the best way to prevent engines from crawling your websites shopping cart, wishlist, log in pags, ect... Obviously have it in robots.txt but is their any other form of action that should be done?
On-Page Optimization | | Romancing0