Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
-
Client site is 15 + pages. New site had noindex/nofollow removed prior to last two crawls.
-
Okay I just did a test crawl with 'Screaming Frog' SEO spider - no issues
Robots.txt - not blocking it.
Headers - not blocking them.
Therefore definitely look at Keri's post as it's something SEOmoz's end, so check out that thread for some possible answers
-
Sure. The site is bayareadrainage.com . Thanks in advance.
-
If GA means Google Analytics, Google Analytics doesn't crawl pages -- it reports visits to pages that have the GA code on them. You can have the content behind a password and blocked in robots.txt and if GA code is there, it would still track people.
https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages lists some common reasons why only one page is crawled. If you still have problems with crawling, send a note to [email protected] and we'll help you figure things out.
-
Are you able to post the URL? That way I, or another Mozzer could check the crawlability and just check you're not accidentally blocking robots another way?
I would also check your robots.txt to make sure you're not Disallowing all your site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Getting high priority issue for our xxx.com and xxx.com/home as duplicate pages and duplicate page titles can't seem to find anything that needs to be corrected, what might I be missing?
I am getting high priority issue for our xxx.com and xxx.com/home as reporting both duplicate pages and duplicate page titles on crawl results, I can't seem to find anything that needs to be corrected, what am I be missing? Has anyone else had a similar issue, how was it corrected?
Technical SEO | | tgwebmaster0 -
Is there a way to index important pages manually or to make sure a certain page will get indexed in a short period of time??
Hi There! The problem I'm having is that certain pages are waiting already three months to be indexed. They even have several backlinks. Is it normal to have to wait more than three months before these pages get an indexation? Is there anything i can do to make sure these page will get an indexation soon? Greetings Bob
Technical SEO | | rijwielcashencarry0400 -
Cant find internal links on one of my pages.
When I run open site explorer for www.kingremodeling.com/ss.php?pid=5 it says there are 40 links to it on my site. However, I cannot find these links on any of the pages that open site explorer lists such as my homepage www.KingRemodeling.com. Totally confused!
Technical SEO | | allb830 -
Should I nofollow Geo-located links on a site?
I run various sites that use Geo-location to place related links in navigation menus on a page. For example, if you land on the home page, we will see that you are in Florida and then in one of the content boxes on the page, show job listings that this site has in Florida. We also give the option to search for other jobs or use other navigation options. The idea is to try to help the user along the best we can, but ..... What opinions do persons have here on if these links should be nofollowed as GoogleBot will always see links to places in California etc. - wherever Googlebot is crawling from? Would this then be confusing as we are a site that focused on the entire US and not just California etc Thanks!
Technical SEO | | CleverPhD0 -
Job/Blog Pages and rel=canonical
Hi, I know there are several questions and articles concerning the rel=canonical on SEOmoz, but I didn't find the answer I was looking for... We have some job pages, URLs are: /jobs and then jobs/2, jobs/3 etc.. Our blog pages follow the same: /blog, /blog2, /blog/3... Our CMS is self-produced, and every job/blog-page has the same title tag. According to SEOmoz (and the Webmaster Tools), we have a lots of duplicate title tags because of this problem. If we put the rel=canonical on each page's source code, the title tag problem will be solved for google, right? Because they will just display the /job and /blog main page. That would be great because we dont want 40 blog pages in the index. My concern (a stupid question, but I am not sure): if we put the rel=canonical on the pages, does google crawl them and index our job links? We want to keep our rankings for our job offers on pages 2-xxx. More simple: will we find our job offers on jobs/2, jobs/3... in google, if these pages have the rel=canonical on them? AND ONE MORE: does the SEOmoz bot also follow the rel=canonical and then reduce the number of duplicate title-tags in the campaigns??? Thanx........
Technical SEO | | accessKellyOCG0 -
Is any know if seomoz update for site crawl.
i belive my site www.breeze-air.com hit by penguin; i found that i had un-natural anchors text and able to remove around 1200 from the 1900 seomoz found. seomoz still shows those anchors - but when i check the link its not there. i removed them 3-4 weeks ago any idea?
Technical SEO | | eoberlender0 -
When Is It Good To Redirect Pages on Your Site to Another Page?
Suppose you have a page on your site that discusses a topic that is similar to another page but targets a different keyword phrase. The page has medium quality content, no inbound links, and the attracts little traffic. Should you 301 redirect the page to a stronger page?
Technical SEO | | ProjectLabs1 -
Noindex, follow duplicate pages
I have a series of websites that all feature a library of the same content. These pages don't make up the majority of the sites content, maybe 10-15% of the total pages. Most of our clients won't take the time to rewrite the content, but it's valuable to their site. So I decided to noindex, follow all of the pages. Outside of convincing them all to write their own versions of the content, is this the best method? I could also block the pages with robots.txt, but then I couldn't pass any link juice through the pages. Any thoughts?
Technical SEO | | vforvinnie0