Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My domain authority is low in MOZ and high in other apps
_I checked my site in other SEO apps and they are showing a higher DA for my site. However, in MOZ is very low. Can you please explain it to me? _what caused the difference?
Moz Pro | | K_Monestel0 -
MOZ not accepting recent changes
We have made some changes on our website according to the SEO standards per Moz report and did some crawler tests but it seems that MOZ is not crawling our recent changes. It is showing the results according to the previous data. What can be the problem here?
Moz Pro | | torbett0 -
Is there a tool on Moz or out on the internet that does bad link checker
I'm still pretty new to this and I was wondering if there is a free software, one of Moz or free out on the the internet that allows you to check bad links. I've done a lot of link building with citations and directories that for my clients industry. I just don't want to add their website and profile to a bad/risky directory and it penalizes my clients. I've seen a few out there, but I need one that is respectable and reliable. Any suggestions? I found one called bad neighborhood http://www.bad-neighborhood.com/text-link-tool.htm. Thanks Again, Benny
Moz Pro | | ACann0 -
Moz Website Grader Export Omits Check Graphics
So, I love the Grader, but when I export to PDF the pretty little graphics go away. Is this correctable?
Moz Pro | | ClickMonsterIM0 -
What are the factors that have an impact on the google search results?
When I compare the ranking of my keywords with the Moz Rankings and my own search results I get different results even I use the google chrome incognito browser (not logged in, no cookies, no cache) and it's definitely the same country-setting (google.de) where the search is run. I double checked it with several browsers and computers, and I still get a different result as the Moz Ranking Tracker. The keyword is "schafkopf" and the site is www.sauspiel.de. The relevant country is Germany (www.google.de). Does anyone know the reason why the results differ? Could it be the search query was made from different countries/IPs? Any ideas, hints?
Moz Pro | | sauspiel0 -
SEO MOZ keyword ranking tool
Hi Guys, Can anyone please help me! I have a client that has two web pages that he wants optimising, one page is the home page and the other page is within the page.. Does anyone know if the SEO MOZ ranking tool only looks at the home page for jeywords or will it look at what ranking is the the other page within the website?? Or do I have to set up separate campaign jut for one page within the website?? Can anyone answer this please? Thanks Gareth
Moz Pro | | GAZ090 -
Www.mydomain.tld same as mydomain.tld - unexpected results
Hi I set up a campaign to monitor a website and chose to track the root domain, however the report showed every single page on my website was duplicated with itself when seen as www.mydomain.tld as opposed to just mydomain.tld. I deleted the campaign and set it up to monitor only the www.mydomain.tld as I don't use any other subdomains, however in Bing results the website is listed without the www and so is not recognised in the ranking checker. How can I set it up so that seomoz recognises the pages as the same either with or without the www?
Moz Pro | | PandyLegend0