How to find 20 hidden 404s
-
Hello,
We have like twenty 404s left to find. How do you find these when:
1. They don't show up in Google Webmaster Tools
2. They don't have any other internal or external pages linking to them.
3. They don't show up in site:domain.com (We have 9000 pages and only 600 show up - I fixed those out of the 600).
4. They are probably causing high bounce rates.
5. They're not in the sitemap
Thanks!
-
You should be able to crawl the entire site without increasing the RAM, once you buy the paid version.
-
Right in the paid version wont stop! You can see in their website that the limit of urls will be remove once you buy the license. http://www.screamingfrog.co.uk/seo-spider/licence/
It should catch everything, maybe you can contact their support just to make sure, they are very good on support over twitter.
-
Yes, but if it only check 397 in the free version is it going to stop there in the paid version as well. Just making sure.
Also, will it catch everything?
-
Yes I just run a craw for 500 000 urls
-
The free version checked 397 URLs. Will the purchased version check all 9000?
-
You can try crawl the site using screamingfrog, make sure you check the box “Check Links Outside Folder” in the spider menu.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
CSS Hidden DIVs - not collapsable content. Amber light?
I'm in the planning stage of a new ecommerce page. To reduce duplication issues, my page will be static with 20% of the page compiled of dynamic fields. So when a user selects a size, or color, the dynamic fields are the only ones that change as the rest of the content is the same. I can keep a static URL and not worry about duplication issues. Focus can be on strengthening this single URL with rich schema, reviews, and backlinks. We're going to cache a default page so for crawlers, the dynamic field doesn't appear empty. My developer said they can cache the page with all the variants of the dynamic fields, and use hidden DIVs to hide them from the user. This way, the load speed can be high, and search engines might crawl those keywords too. I'm thinking about and going.."wait a minute, that's a good idea..but would a search engine think I am hidding content and give me a penalty?". The hidden content is relevant to the page and it only appears according to the drop down to make the user experience more "friendly". What do you think? Use hidden DIV or use javascript to not allow bots to crawl the hidden data at all?
Intermediate & Advanced SEO | | Bio-RadAbs0 -
Can't find X-Robots tag!
Hi all. I've been checking out http://www.unthankbooks.com/ as it seems to have some indexing problems. I ran a server header check, and got a 200 response. However, it also shows the following: X-Robots-Tag:
Intermediate & Advanced SEO | | Blink-SEO
noindex, nofollow It's not in the page HTML though. Could it be being picked up from somewhere else?0 -
How do you find a truely knowledgable SEO person to analyze are large site?
We are a large site, 5600 pages with local pages in almost every city across the US. We are struggling with page rank on some pages and I dont think its as simple as backlinks and its definitely not poor on-page SEO. I think we might have some truly technical issues that is causing us to get penalized in SERP's. Any agencies which analyze sites? This is NOT a job posting so please don't send me messages...I truly want to know how/where to find a solution to our problem. Thanks
Intermediate & Advanced SEO | | CTSupp0 -
Why is Google Webmaster Tools reporting a massive increase in 404s?
Several weeks back, we launched a new website, replacing a legacy system moving it to a new server. With the site transition, webroke some of the old URLs, but it didn't seem to be too much concern. We blocked ones I knew should be blocked in robots.txt, 301 redirected as much duplicate data and used canonical tags as far as I could (which is still an ongoing process), and simply returned 404 for any others that should have never really been there. For the last months, I've been monitoring the 404s Google reports in Web Master Tootls (WMT) and while we had a few hundred due to the gradual removal duplicate data, I wasn't too concerned. I've been generating updated sitemaps for Google multiple times a week with any updated URLs. Then WMT started to report a massive increase in 404s, somewhere around 25,000 404s per day (making it impossible for me to keep up). The sitemap.xml has new URL only but it seems that Google still uses the old sitemap from before the launch. The reported sources of 404s (in WMT) don't exist anylonger. They all are coming from the old site. I attached a screenshot showing the drastic increase in 404s. What could possibly cause this problem? wmt-massive-404s.png
Intermediate & Advanced SEO | | sonetseo0 -
How to find all of a website's SERPs?
Was wondering how easiest to find all of a website's existing SERPs?
Intermediate & Advanced SEO | | McTaggart0 -
How to remove "Results 1 - 20 of 47" from Google SERP Snippet
We are trying to optimise our SERP snippet in Google to increase CTR, but we have this horrid "Results 1 - 20 of 47" in the description. We feel this gets in the way of the message and so wish to remove it, but how?? Any ideas apart from removing the paging from the page?
Intermediate & Advanced SEO | | speedyseo0 -
How to find the number of DoFollow links on the homepage
Dear SEOmozzers, I just have a quick question. Can you please tell me how I can find out how many DoFollow links I have on the page of a site? I did a link exchange with a person who claims that she found 50+ DoFollow links on my homepage. How did she do that? What tool did she use to find out the number of DoFollow links on my homepage? Thank you. Sal
Intermediate & Advanced SEO | | salvyy0 -
How the hell do you find good PR websites?
Hi Guys Can someone please help! How do you find high relevant PR sites that can be easiliy located using the SEO MOZ tools? How can I best use the SEO MOZ tools to locate good websites that will help me with linking building? Are the SEO MOZ tools any good?? Please can you help guys..?? Thanks Gareth
Intermediate & Advanced SEO | | GAZ090