Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Robots.txt blocking Moz
Moz are reporting the robots.txt file is blocking them from crawling one of our websites. But as far as we can see this file is exactly the same as the robots.txt files on other websites that Moz is crawling without problems. We have never come up against this before, even with this site. Our stats show Rogerbot attempting to crawl our site, but it receives a 404 error. Can anyone enlighten us to the problem please? http://www.wychwoodflooring.com -Christina
Moz Pro | | ChristinaRadisic0 -
Duplicate content flagged by Moz that's not actually duplicate content at all
Hi, Moz has flagged a whole lot of pages as dupe content, but I cannot see how they qualify as such.
Moz Pro | | Caro-O
Not sure if I'm allowed to post actual URLs here....happy to if I can, but I feel certain that the pages are not 90% similar. Has anyone else had this experience? ~Caro1 -
Help us with Moz Academy!
Hey Q&Aers! I just wanted to create a discussion thread here to talk solicit some feedback and help with Moz Academy! We want your help coming up with future lessons so we’re getting you the content you need. So, think about what you’d or your team would like to learn from Rand and others at Moz. Let us know! “So, what is Moz Academy?” you may ask. Well, Moz Academy gives subscribers the power to be better marketers, which will enable them to use Moz’s products with more depth and awesomeness. We want to provide a hub of marketing knowledge that will create a stronger community where people can teach each other while using Academy as a frame of reference. The key to this project is empowering you to kick even more ass than you already do! If you have any questions about Moz Academy, feedback about lessons, or ideas on a future curriculum please feel free to get involved in the discussion below. Looking forward to see what everyone comes up with.
Moz Pro | | Nick_Sayers1 -
Couple of Moz's bugs
There are still some bugs in a new Moz: I can still find a lot of mentions of "SEOMoz", for example: in a footer, in a Q/A form (SEOMoz Resources), in a new question form (SEOmoz PRO Application, SEOmoz Tools, etc.) On a main form (http://moz.com/pro/home) sometimes my full name is not visible at all, sometimes my MozPoints are hidden (on a left top corner); There is not direct link from http://devblog.moz.com/ to a main website; Regards.
Moz Pro | | ditoroin0 -
New campain which give exactly same results as a completed old one after 2 hours
I started a campain a long time ago. 10,372 pages have been crawled and I have the results. Great. I started a new campain today (after migration of new code) and after two hours, the campain shows excatly the same results than what the the previous one completed in 4 days, as if it was the same crawl. Then the crawl stops and we see that the next one is in 7 days... Before there was a way to launch a campain after code modification to see right away the benefits. How can we do to see ifi the changes brought something without waiting for the next crawl? Thank you.
Moz Pro | | celyfred0 -
Help Understanding Crawl results on this site
I'm just starting to SEO this site http://thefirmbusinessbrokerage.com/welcome and I'm having trouble with the crawl report data. First question, should I be building links to the site above or the main page http://thefirmbusinessbrokerage.com/ (which is a flash intro). If I build links to the flash page, what do I do to the forwarding URL to the welcome page to make it effective? Second question, why does the crawl data report show up almost completely blank? Is this site perfect or are there some onsite issues that I'm not seeing. Thanks for your support and guidance on this site. I'm not hosting the site, just building links and offering optimization advice onsite. JOE
Moz Pro | | KreativElement0 -
Keyword Difficulty Tool on local results. Is this really working?
I've tried the Keyword Difficulty Tool on "consultant seo" in Romania (that is .ro) The tool says that it's Highly Competitive (59%), but I've got my website on #8 in 2 weeks. Question: Is this tool really working for local results? It seems way off... Actually I was kind of expecting for this keyword to be easy, what is confusing is the "59%" from the Keyword Difficulty results. Thanks for the answers!
Moz Pro | | wizant0
