Why is the SEOmoz crawler crawling the old version of our website?
-
Hello,
I'm a new SEOmoz member.
On Dec. 2nd, after completely redesigning our website, we migrated to a new hosting company by switching our DNS to the new server.
The vast majority of the URLs have changed and we configured redirects of the old URLs to the new ones. Although, this task is not completed yet.
After the migration, I created an account on SEOmoz to be able to track our progress and find the issues to fix to optimize our SEO.
For some reason, in the SEOmoz reports it is the old URLs that show up. Unless the crawler does not actually crawl the pages and only uses the indexed pages to generate its report, I don't understand how could this possible.
Anyone has a clue?
When will the new URLs be indexed by SEOmoz and the major search engines?
Thanks for your help!
-
Thanks for the answers guys.
@Robert: It is indeed the initial report. Yet I did receive other reports since. Also thanks for suggestion the Screaming Frog tool. The report I get from it will be useful in my discussion with our CRM provider.
@Cyrus: I also used URI Valet. I don't understand, the 302 redirects are on picture links. I'll communicate with our CRM provider about that. Also, I don't understand how any of our old URLs could be reached crawling our website as this is a completely new website. No part was transferred over so the shouldn't be any way for the crawler to see our old URLs.
Thanks again
-
The only way that SEOmoz will crawl your old URLs is if that is what your web servers is delivering to the SEOmoz crawlers. In this case, you might want to make sure all your redirect are in place.
I did a quick check of the URL listed in your public profile using the tool URI valet, and found at least 4 redirects from the homepage, including a 302, which passes no link juice. This is definitely something you want to look into.
Generally the SEOmoz robots will crawl your site once a week, so 99% of the time you will see new infomation contained in your campaign within 7-10 day. If you suspect an error, be sure to contact the help team ([email protected])
Also, here's my favorite article on site migration. Although you didn't migrate a complete site, many of the principals still apply.
https://seogadget.co.uk/surviving-seo-site-migration/
Best of luck with your SEO!
-
Gestisoft-QC
First, SEOmoz does crawl the site it is not using indexed pages. I am assuming you set this up as a campaign and got the initial report and saw the old urls? If you are looking for links, OSE is different and does take roughly 3 to 7 weeks to show due to the amount of work involved (they are crawling the web as opposed to a site for this).
I ran your site through Screaming Frog and see the 301's, etc. You might try using Screaming Frog to see the Header Responses, etc. http://www.screamingfrog.co.uk/seo-spider/ Screaming Frog has a free download for windows or Mac and that should be sufficient to see what URLs are showing, etc. It will not give you the other info you get from moz.
I cannot see where the moz would be different, but this should at least give you a good cross reference.
If you don't get resolution there, here is the link to helpful info on moz Pro: http://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-only-crawled-one-page
You can also ask for help via [email protected]. They are very responsive.
Hope this helps
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate title on Crawl
Ok, this should hopefully be a simple one. Not sure if this is a Moz crawl issue of redirect issue. Moz is reporting duplicate title for www.site.co.uk , site.co.uk and www.site.co.uk/home.aspx is this a canonical change or a moz setting I need to get this number lower.
Moz Pro | | smartcow0 -
Why seomoz crawler does not see my snapshot?
I have a web app that uses angularJS and the content is all dynamic (SPA). I have generated snapshots for the pages and write a rule to redirect ( 301) to the snapshot in case of find escaped_fragment in the URL. E.g http://plure.com/#!/imoveis/venda/rj/rio-de-janeiro Request: http://plure.com/?escaped_fragment=/imoveis/venda/rj/rio-de-janeiro is redirected to: http://plure.com/snapshots/imoveis/venda/rj/rio-de-janeiro/ The snapshot is a headless page generated by PhantomJS. Even following the guideline ( https://developers.google.com/webmasters/ajax-crawling/docs/specification) I still can't see my page crawled and I also in SEOMoz I can only see the 1st page crawled with no dynamic content on it. Am I doing something wrong? SEOMoz was supposed to get the snapshot based on same rules of GoogleBot or SEOMoz does not get snapshots?
Moz Pro | | plure_seo0 -
SEOmoz Toolbar - Am I the only one who figured this out?
I have been using the SEOmoz toolbar for a while. In the course of a day, and with so many projects going on I end up having about 15-20 tabs in Firefox open at once. Because of the toolbar, my browsing session slows to a crawl and I hate to disable the toolbar because then I have to shut down the 10 projects I am working on. I usually close down my browser when I can't take it anymore and it feels like I am browsing on my 1990's palm device. What I decided to do is install the toolbar on Google Chrome. I don't use this browser very often so now whenever I need to check out a website, I just use Chrome instead of Firefox. Maybe there is a way to turn off the toolbar that I don't know about. Any other toolbar tips?
Moz Pro | | Czubmeister0 -
Campaign Crawl Report
Hello, Just a quicky, is there anyway I can do a crawl report for something in a campaign so I can compare the changes? I know you can do a separate crawl test, but it wont show the differences,and the next crawl date isnt untill the 28th.
Moz Pro | | Prestige-SEO0 -
Only one page has been crawled
I am running a campaing for three weeks now and first two crawls was ok but the last one is showing only one page crawled. the subdomain I am tracking is: www.cubaenmiami.com I have everything correct in my site. Regards Alex
Moz Pro | | esencia0 -
Is the SEOMoz Rank Tracker still accurate?
I monitor several keywords on several domains and i have found that rank tracker gives me one result yet when searching via Google myself (on different machines) i get a different result. I've tried several devices on different networks so there is no chance that Google is remembering my searches as such and i ensure that i am always signed out of my account. All devices provide the same results yet Rank Tracker provides a completely different result. It's as though Rank Tracker is a day ahead or something. Can anyone shed any light for me please?
Moz Pro | | MarkHincks0 -
Can I change the crawl day ?
Hi All I hope there is a simple solution to this - we have a number of campaigns setup which are all crawled, and therefore updated, on different days of the week. We review these weekly and it would be much easier if they were all crawled on the same day. Is it possible to change the crawl day for some campaigns? Thanks Roy
Moz Pro | | bluelogic0 -
What causes Crawl Diagnostics Processing Errors in seomoz campaign?
I'm getting the following error when seomoz tries to spider my site: First Crawl in Progress! Processing Issues for 671 pages Started: Apr. 23rd, 2011 Here is the robots.txt data from the site: Disallow ALL BOTS for image directories and JPEG files. User-agent: * Disallow: /stats/ Disallow: /images/ Disallow: /newspictures/ Disallow: /pdfs/ Disallow: /propbig/ Disallow: /propsmall/ Disallow: /*.jpg$ Any ideas on how to get around this would be appreciated 🙂
Moz Pro | | cmaddison0