URLs dropping from index (Crawled, currently not indexed)
-
I've noticed that some of our URLs have recently dropped completely out of Google's index.
When carrying out a URL inspection in GSC, it comes up with 'Crawled, currently not indexed'.
Strangely, I've also noticed that under referring page it says 'None detected', which is definitely not the case.
I wonder if it could be something to do with the following? https://www.seroundtable.com/google-ranking-index-drop-30192.html - It seems to be a bug affecting quite a few people.
Here are a few examples of the URLs that have gone missing:
https://www.ihasco.co.uk/courses/detail/sexual-harassment-awareness-training
https://www.ihasco.co.uk/courses/detail/conflict-resolution-training
https://www.ihasco.co.uk/courses/detail/prevent-duty-training
Any help here would be massively appreciated!
-
I don't know why but I am facing the same issue from past 3 monts.
My most of the URLs are getting de indexed after indexing the search console. -
@kingshah001 said in URLs dropping from index (Crawled, currently not indexed):
Same issue here, my most of the URLs are getting de indexed after indexing the search console.
Same issue here, my most of the URLs are getting de indexed after indexing the search console.
https://inshotproapps.com
https://instoproapps.com/inshot-for-pc/ -
Thanks for sharing details.
-
Same issue here, my most of the URLs are getting de indexed after indexing the search console.
Some of the URLs are below:
https://apkcroc.com/
https://apkcroc.com/vn-mod-apk/
https://apkcroc.com/kinemaster-mod-apk/
https://apkcroc.com/terragenesis-mod-apk/
https://apkcroc.com/sky-fighters-3d-mod-apk/ -
My site is also a victim of same the issue, collecting bits and actionable advice. I'm planning to post my experience on Moz forum soon.
-
Hello,
since the beginning of ladykiller.nl I am having the same issues with Google to crawl sitemap(s) and index urls. I am using Yoast as a plugin for the sitemap.
For the moment +3620 urls are indexed, but my website has +10.000 urls :(.
Also from time to time I get a notice in GSC that Google can not fetch certain sitemap urls f.e. https://ladykiller.nl/post-sitemap.xml. Mostly the issue is fixed after a week or so. Please find print screen here: https://prnt.sc/Pm_h2Arjxu-kAlready asked on numerous forums for help, as I can not find a solution to get this problem fixed. However, without any good results so far.
Therefore, I am trying it here again in the hope maybe some of you guys have some better understanding of what the issue might be and how it can be fixed. All help is highly appreciated!
Thanks in advance for having a look into it :)!
Warm regards,
John -
Hi there,
The third URL you are referencing, is actually indexed:
https://dmitrii-regexseo.tinytake.com/tt/NDY4NDY4N18xNDgzNjgzMA
As for "crawled, not indexed" - in most cases it happens because of one and only reason - Google is seeing your page as thin content, not worth being indexed. Typically it happens on bigger sites with a lot of similar pages. In your case, you got many courses, with exactly same structure. So, if the content is not completely different, then Google might deem it not worthy.
As for the bug you referenced - did your URLs drop off the index exactly at the time when this issue has been discovered? (aka within the last week?).
Do you have any cannibalization happening?
To me it looks like that's the case. If I do this search: "site:https://www.ihasco.co.uk/ Sexual Harassment Training course"
There are many pages that are indexed and are ranking: https://dmitrii-regexseo.tinytake.com/tt/NDY4NDcwN18xNDgzNjg4Mg
So, basically, you have pages that are more authoritative with similar content. Therefore your courses pages are dropping as thin content.
I would recommend doing some internal linking optimization to tell Google what is actually important. Look in GSC for internal links metrics.
Hope this helps.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google crawl drop
the crawl request of my company site: https://www.dhgate.com/ has dropped nearly over 95%, from daily 6463599 requests to 476493 requests at 12:00am on 9th, Oct (GMT+8). This dramatic dropping trend not only showed in our GSC crawl stats report but also our company's own log report. We have no idea what’s going on. We want to know whether there is an update of google about crawlling, or is this the issue of our own site? If something is wrong with our site, in what aspects would you recommend us to check, analyze and accordingly optimize?
Technical SEO | | DHgate_20140 -
Large Drop in Indexed Pages But Increase in Traffic
We run a directory site and noticed about a week ago that Google Webmaster Tools was reporting a huge drop in indexed pages (from around 150,000 down to 30,000). In the same time, however, our traffic has increased. Has anyone seen this before or have any ideas on why this could happen? I have search for technical errors but nothing has changed on our site or our content.
Technical SEO | | sa_787040 -
No Index PDFs
Our products have about 4 PDFs a piece, which really inflates our indexed pages. I was wondering if I could add REL=No Index to the PDF's URL? All of the files are on a file server, so they are embedded with links on our product pages. I know I could add a No Follow attribute, but I was wondering if any one knew if the No Index would work the same or if that is even possible. Thanks!
Technical SEO | | MonicaOConnor0 -
Should I make a new URL just so it can include a target keyword, then 301 redirect the old URL?
This is for an ecommerce site, and the company I'm working with has started selling a new line of products they want to promote.Should I make a new URL just so it can include a target keyword, then 301 redirect the old URL? One of my concerns is losing a little bit of link value from redirecting. Thank you for reading!
Technical SEO | | DA20130 -
Website is not indexed in Google
Hi Guys, I have a problem with a website from a customer. His website is not indexed in Google (except for the homepage). I could not find anything that can possibly be the cause. I already checked the robots.txt, sitemap, and plugins on the website. In the HTML code i also couldn't find anything which makes indexing harder than usual. This is the website i am talking about: http://www.xxxx.nl/ (Dutch) The only thing that i am guessing now is the Google sandbox, but even that is quite unlikely. I hope you guys discover something i could not find! Thanks in advance 🙂
Technical SEO | | B.Great0 -
Wordpress URL weirdness - why is google registering non-pretty URLS?
I've noticed in my stats that google is indexing some non-pretty URLs from my wordpress-based blog.
Technical SEO | | peterdbaron
For instance, this URL is appearing google search: http://www.admissionsquest.com/onboardingschools/index.php?p=439 It should be: http://www.admissionsquest.com/onboardingschools/2009/01/do-american-boarding-schools-face-growing-international-competition.html Last week I added the plugin Redirection in order to consolidate categories & tags. Any chance that this has something to do with it? Recs on how to solve this? Fyi - I've been using pretty URLS with wordpress from the very beginning and this is the first time that I've seen this issue. Thanks in advance for your help!0 -
Will a drop in indexed pages significantly affect Google rankings?
I am doing some research into why we were bumped from Google's first page into the 3rd, fourth and fifth pages in June of 2010. I always suspected Caffeine, but I just came across some data that indicates a drop in indexed pages from 510 in January of that year to 133 by June. I'm not sure what happened but I believe our blog pages were de-indexed somehow. What I want to know is could that significant drop in indexed pages have had an effect on our rankings at that time? We are back up to over 500 indexed pages, but have not fully recovered our first page positions.
Technical SEO | | rdreich490 -
PageRank Dropped?
The Symptoms About a year ago, our site EZWatch-Security-Cameras.com had a PageRank of 5. Several months ago it sunk to a 4 and we were a little worried, but it wasn't anything to really sweat over. At the end of january we noticed it had dropped again to a PR3, again we were a little more worried. When the farmer update hit we suddenly dropped to a PR1 but our traffic wasn't seriously affected, and in march most of the pages regained their PageRank. I noticed this morning that our homepage rank has once again dropped to a PR1. I am waiting to see if there has been any significant drop in traffic, but I haven't spotted anything that stands out significant, aside from an increase in the average cost for our paid search account of about 5%. The Problems We've Spotted Keep in mind that our current website is fairly old (2005) and we are ready to launch a new one. Our current website is running on X-Cart, and we have a few modules added on. Problem 1 - One such module handles a custom kit builder, this area has not been restricted by crawlers and it could be generating a large amount of needless page crawls. Problem 2 - Another module allows "SEO friendly URL's" according to the developer, but what actually happens is a visitor could type in any-url-they-like-for-product-id**-p-11111.html**, where the underlined section is any character string (or lack of), followed by either a product or category indicator and the id for said item. This causes a massive amount of virtual page duplications, and the module is encrypted so we aren't able to modify it to include rel="canonical" tags. Obviously this causes massive amounts of seemingly duplicate content. Problem 3 - In addition to the regular URL duplication, we also recently acquired the domain EZWatch.com (our brand name, easier to remember). That domain name responds with the content from our regular website, and it will be the primary domain name when we change shopping carts. With the second domain name the content could also be considered a duplication. The Solutions We're Working On The website we use was designed in 2005, and we believe that it's reached the end of its useful life. Over the past several months we have been working on an entirely new shopping cart platform, designed from the ground up to be more efficient operationally-speaking, and to provide more SEO control. The new site will be ready to launch within days, and we will start using the new Domain name at the same time. We are planning on doing page-to-page301 redirects for all pages with at least 1 visit within the past 180+ days, according to our Google Analytics reports. We are also including rel="canonical" on all pages. We will also be restricting dynamic sections of our website via the robots.txt file. So What More Can We Do? With your collective SEO experience, what other factors could also be contributing to this decline?
Technical SEO | | EZWatchPro0