Crawl errors for pages that no longer exist
-
Hey folks,
I've been working on a site recently where I took a bunch of old, outdated pages down. In the Google Search Console "Crawl Errors" section, I've started seeing a bunch of "Not Found" errors for those pages. That makes perfect sense.
The thing that I'm confused about is that the "Linked From" list only shows a sitemap that I ALSO took down. Alternatively, some of them list other old, removed pages in the "Linked From" list.
Is there a reason that Google is trying to inform me that pages/sitemaps that don't exist are somehow still linking to other pages that don't exist? And is this ultimately something I should be concerned about?
Thanks!
-
Thanks for the question, this can definitely be annoying for webmasters!
Unfortunately, bots can don't everything in parallel. They have to take steps...
Step 1. Take List #1 of links.
Step 2. Crawl those links and build List #2.
Step 3. Crawl List #3 and build List #4...Now, sometimes it doesn't follow that same order. Let's say that in Step 3 it finds a bunch of pages with unique content. Maybe the next time around, it goes and checks some of those links in Step 3 without first checking if they were still linked. Why start the crawl all the way from the beginning again when you have a big list of URLs?
But, this creates a problem. When some of those links it crawled in Step 3 aren't there any more, Google will tell you they aren't there and tell you how they originally found them (which happened to be from a page in List #1). But what if Google hasn't checked that link in List #1 recently? What if you just removed it too?
Well, for a little while, at least, you will end up with errors.
Now, here comes the real rub - how long will it take for Google to find and correct that message it left you in the crawl report? Days? Weeks? Months? Who knows. Your best bet is to mark them as fixed and force Google to keep rechecking. Eventually, they will figure it out.
TL;DR; it is a data freshness and reporting issue that isn't your fault and isn't worth your time.
-
No - Google is just showing how slow it is when updating data in Webmaster tools.
Don't worry - if you wait long enough they'll go away. You could also mark them as solved (do this only if you are sure that there are no links pointing to these pages - to check if your internal linking is ok Screaming Frog is great tool)
Dirk
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does intercom support pages and redirect issue can affect the SEO performance of our website?
I noticed that in the redirect issues I have, most of the issues are coming from our Intercom support links. I want to ask, does intercom support pages and redirect issue can affect the SEO performance of our website?
Reporting & Analytics | | Envoke-Marketing0 -
Google Analytics Landing Page Report Discrepancy
I have noticed that when I run a landing page report and use the advanced option so I can view only the landing pages that include a particular string in the URL, have noticed that I in the report, the graph at the top will say one thing, but the data below says something else. For example, the graph for one particular search shows 200 Impressions, but the info below says 700 impressions and 610 clicks. Anyone seen anything similar or have any ideas why? Thanks! Craig
Reporting & Analytics | | TheCraig0 -
Switch to www from non www preference negatively hit # pages indexed
I have a client whose site did not use the www preference but rather the non www form of the url. We were having trouble seeing some high quality inlinks and I wondered if the redirect to the non www site from the links was making it hard for us to track. After some reading, it seemed we should be using the www version for better SEO anyway so I made a change on Monday but had a major hit to the number of pages being indexed by Thursday. Freaking me out mildly. What are people's thoughts? I think I should roll back the www change asap - or am I jumping the gun?
Reporting & Analytics | | BrigitteMN0 -
2 days in the past week Google has crawled 10x the average pages crawled per day. What does this mean?
For the past 3 months my site www.dlawlesshardware.com has had an average of about 400 pages crawled per day by google. We have just over 6,000 indexed pages. However, twice in the last week, Google crawled an enormous percentage of my site. After averaging 400 pages crawled for the last 3 months, the last 4 days of crawl stats say the following. 2/1 - 4,373 pages crawled 2/2 - 367 pages crawled 2/3 - 4,777 pages crawled 2/4 - 437 pages crawled What is the deal with these enormous spike in pages crawled per day? Of course, there are also corresponding spikes in kilobytes downloaded per day. Essentially, Google averages crawling about 6% of my site a day. But twice in the last week, Google decided to crawl just under 80% of my site. Has this happened to anyone else? Any ideas? I have literally no idea what this means and I haven't found anyone else with the same problem. Only people complaining about massive DROPS in pages crawled per day. Here is a screenshot from Webmaster Tools: http://imgur.com/kpnQ8EP The drop in time spent downloading a page corresponded exactly to an improvement in our CSS. So that probably doesn't need to be considered, although I'm up for any theories from anyone about anything.
Reporting & Analytics | | dellcos0 -
Page Retirement
I have a site with 6000 indexed urls. 1,500 have traffic I feel is valuable and 4,500 with almost no traffic (perhaps less than 10 page views in a year). These 4500 are inedxed but have 1 or less in bound links. If I retire the pages, will I help or hurt my Domain Authority and separately my rankings that could produce traffic? I'd appreciate any consideration. Jeffrey Strassman www.consultant360.com
Reporting & Analytics | | biggieshaws0 -
Google Analytics Title tag vs landing page visitors numbers
Hi folks, Just wondering if anyone has any ideas as to why im getting different results in Google analytics. I'm using the Content Efficiency Analysis Report from http://www.kaushik.net which is absolutely awesome. When I search via my title tag I get 920 Unique Visitors over the month but when I search via the landing page URL with the same title tag I get 28. Any ideas to why their should be such a difference. I've also noticed that on that page i'm also getting a Rel Cononical TRUE using a site crawl. Any ideas are much appreciated
Reporting & Analytics | | acs1110 -
Time on page: What happens when I open many tabs?
Hello everyone, I was studying Analytics, and checked that the time on page is calculated by the diference of the time you entered the page and when you click to go to another one. But how the time is calculated when I open several links using new tabs in different moments? Does Google counts the last tab? Just a guess... Thanks!
Reporting & Analytics | | seomasterbrasil0 -
4xx errors
Hi I checked in my campaign to look for errors on my page and i have got a report showing me a lot of 404 broken or dead links error. So how can i view the source of the broken link in order to fix it. Thank you!
Reporting & Analytics | | DigitalDave0