Moz crawling doesn't show all of my pages
-
Hello,
I'm trying to make an SEO on-pages audit of my website.
When using the SEO crawl, i see only one page while i have much more pages in this website (200+). Is it an issue with the sitemap ? The robot.txt ?
How can i check and correct this. My website is discoolver.com
-
Hi there,
Sam from Moz's Help Team here!
So, I took a look at your site, and unfortunately I believe this is a result of our crawler's inability to work with Javascript. Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server.
Our crawler works by parsing the source code of your site, looking at HTML elements. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!
-
Thanks for the answers.
I believe the issue is about a rel=canonical from the homepage which links to an external website...
Can this tag prevent a good website crawl ?
-
So just looking it appears you don't have a robots.txt file or a sitemap. At least one that I could easily find.
This Moz post might help when creating one.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Links to non-exisiting pages
Hi, I have tons of incoming links to target pages on my website that do not exist. If you follow the link you get a 404 error. The anchor text is always a "spammy" non related text. Does anyone know what is happening and how to get rid or block these links? Thanks, Dirk
Link Explorer | | FoodJEt0 -
Error Message on Moz Crawler
Hi all, Just ran into this issue, when analysing this site. Just got this message when using MOZ "Page Optimisation Error". Anyone know why? It seems to be working fine on other SEO analyser tools. Website is: www.sbpcreativemedia.com.au Thanks in advance! luXS8V5
Link Explorer | | Dushala0 -
Why aren't my "page social metrics" increasing?
I post a lot to Facebook & twitter, & my "page social metrics" haven't budged in 4 weeks. I even stopped using bit.ly & stated the full URL as a test. Still no change. The fb account is /getgoodgifts & twitter is /giftsing. Thoughts on why social metrics aren't increasing?
Link Explorer | | giftsing0 -
Get a List of Pages in a Site, Ranked by their PA?
Is there a way to get a list of the pages within one of my sites, and rank it by page authority?
Link Explorer | | JDigitalIdentity1 -
[No Title] in Top Pages using Open Site Explorer
Hello!
Link Explorer | | eagleleather
All of my pages contain properly formatted title tags which display in the browser. However, when I view the 'Top Pages' area of the Open Site Explorer, 99% of those pages show [No Title] in the Page Title & URL column. Does anyone know what might be causing this?0 -
Does OSE crawl our site often? Or do we have an other problem? (internal links)
I am wondering if i can find out when OpenSiteExplorer crawled our entire website for the last time.
Link Explorer | | wilcoXXL
A few months ago we added some internal textlinks to all of our productpages. Those are links to the brandpage of that particular product. Some of our brands do have up to 1500 products, so there should be at least 1500 internal links pointing to that brandpage. But OSE still gives me a wrong count. It says there is only 1 internal link pointing to that brandpage.
Is it because OSE never crawled our site again? Or am i missing something here?(maybe too many internal links to one specific brandpage is not okay?) I'll hope you guys can help me out with this one.0 -
OSE Stats - Number of Unique C IP's and PA/DA keep going up and down. Why?
For the past 3 months (if not more) I have been checking my sites stats on OSE and have noticed drastic decreases and increases in unique C-IP's, ranging from 600 to 1200. I have also noticed the PA & DA going up and down by 5 points. We have a talented team that work hard to market our business ethically with strong content and PR relationships, white-hat link building all the way, active social media and a dedicated on-site technical team. The data from MajesticSEO seems to be a lot more steady and consistent. Why is OSE showing drastic changes in 1 to 2 week intervals? Many thanks Ross
Link Explorer | | David_Connor0