SEOMOZ Crawl Test
-
Guys
I really have an issue that i know have but cannot see if that makes sense.
Basically 3 months ago i did a site wide 301 from economyleasinguk.co.uk to www.economy-car-leasing.co.uk
Every thing looks good get all the correct header responses , all canonicals work perfectly , Google webmaster tools is updated fetch as google bot shows the old site is 301
I tried the seomoz crawl test today on the old domain and got this message
Oh no! Looks like the page you were trying to access is temporarily down
which at first thought ok because the site was not there it wont do it on an old 301 domain, however i tried it on a domain i know has just been 301'd and i got this message
The URL http://www.site1.com/ redirects to http://site2.com/. Do you want to crawl http://site2.com/ instead?
Would you like to:
Continue with www.site1.com
Continue with site2.comI really do not know what to do, its either the redirect script is missing something however its doing what it should or the server is a problem but again its doing what it should so why would SEOMOZ not be able to crawl the old URL like it example site above.
Now the strange thing is Open Site Explorer does see the 301 and asks if i want to check the new URL instead
Ps the redirect is done using PHP redirect which i am asking him to change to a htaccess as its now on a apache server and was wondering if this could be an issue, all pages go to correct pages as requested
Thanks in Advance
-
Hi Robert
Htaccess uploaded, however the code you provided i think was relying on the sites sitting on the same I.P where in fact they are on different servers with different a I.P
so only the bottom line was needed , if we put your code up all looked good but some tools reported 403
Site explorer and site crawler now see the site as 301. so fingers crossed
Just need google to visit the old site and see the change and its not been there for ages , ill just ping it and add it to twitter or something
-
Im sure i can do that for you, especially if your doing it for the good of education and science ..
If you mail me through the site your mail address ill let you have the analytic s too, so as you can see the visitors and the improvements and if all works well it should be a good graph as im currently around 60-70% short of what i had before
This is assuming that what i have already is actually wrong but would make a great topic for some
-
Kelly
Happy to help, but there are strings attached...No, you don't have to let me use the Maserati for a week next time I'm in town...
Before you put the change in effect, please run a measure of all DA/PA for both sites and date it. Then, weekly at least, rerun looking at both. Remember, you will be ask do you want to see the other url so you can still see both even though you have the redirects in place.
Then, you will get a great view of the change over time and can track the effect which, whenever you see a similar question on moz you can refer back to.If you chose to share that data with me, I promise to use the data anonymously and never share what site/sites or page/pages it was from. No plans to publish it in any way as of now, just started tracking sites with 301's about a year ago and like having some that we are not associated with other than tangentially.
Thanks, but no worries if you don't want to share, I understand.
Best
-
Thanks Robert
Thats just a straight forward htaccess 301, makes sense to keep it simple
Again thank you for taking the time in getting back to me
-
Kelly
Here is the code per our dev:
RewriteEngine On
RewriteCond %{HTTP_HOST} ^olddomain.com$ [OR]
RewriteCond %{HTTP_HOST} ^www.olddomain.com$
RewriteRule (.*)$ http://www.newdomain.com/$1 [R=301,L]
This should assist you in getting it done correctly.
He also suggested that the PHP is not the way to go and it does not handle the url to url. So, there you have it.
Best
-
I will get you a version of the 301 As soon as i can to see
Me and my developer had a debate whilst doing this and as he is the expert he kind of won but i still question his thinking
What he did was only send over clean URL's from the old site he insisted on sorting out the canonicalization of the URL along with lower and upper case before the URL was sent so in theory on both servers you have a 301 , we know they are not chaining but if the new site has a 301 capable of resolving any bad URL's that come our way , why have it on both
I wanted just the minimum with less chance for anything to go wrong
If you were doing a name change for a site would you do as Robert Fisher suggests above and do this on a URL to URL basis or would you just do the header response 301 as in your example.
as i do see the benefit in both, but just really need this fixing as business is at an all time low right now
-
Wow
That's really kind of you, i have trouble getting hold of my developer on the weekend so good luck
Many thanks again for your help
-
Fortunately or unfortunately, it is 0400 here in Houston. Fortunately, our web dev magnificent & company VP is still asleep, but works like me (a bit much). I sent an email for the code that will help. He will rise in a couple of hours and see email and direct us. When he does i will send to see if it helps.
Hopefully, we will find the magic juice retriever.
Best,
-
Hi Robert
Many thanks for taking the time to answer
It was just a domain name change
What we did was a URL Change where all the relevant URLS got to their new names in just a couple of lines of code Like in Ians example, I cant post the code yet as its the weekend and my developer is not working
You state i should have done this URL to URL however the sites has 17k pages
I Know In Open site explorer it sees the 301 however when you use the crawl test tool it does not and this is what maid me ask this question again and its that which bothers me the most
My site prior had a DA of 68 and PA of 62 (but even if you check the pages in Google Pr for the home page is showing 1 where it used to be 4 but the inner pages are between 2 and 3 so it just seems the home page is suffering
-
I'd need to see the PHP redirect code. But in CURL it all looks OK.
Make 100% sure that your PHP code is set up with the lower case 'header' command, like this:
header("HTTP/1.1 301 Moved Permanently"); header("Location: http://www.site.com"); ?>
Hope this helps.
-
Kelly,
First, you state: Basically 3 months ago i did a site wide 301 from economyleasinguk.co.uk to www.economy-car-leasing.co.ukBy a sitewide 301 do you mean you simply redirected the domain (meaning one 301 of domain economyleasinguk.co.uk only to www.economy-car-leasing.co.uk only)?
Next, when using OSE and you get the "abc.com redirects to www.abc.com we're showing links to www.abc.com to provide more accurate metrics... That can be from the preferred domain you set in GWMT or the .htaccess file so is likely not a factor in this.
When you do a site wide 301 it must be url to url: abc.com/ to a-b-c.com/ and abc.com/hot-deals to a-b-c.com/todays-sales, or abc.com/our-locations to a-b-c.com/our locations. Otherwise you are wasting all the link juice from the old site. This must be done in the .htaccess file.
To test this- OK I just did and you did it incorrectly and need to back up. When it says "do you want to see the results for economyleasinguk.co.uk instead click on that link for economyleasinguk.co.uk. When you arrive on the page, you have only one result from a site with a domain authority of 50 and that home page has a PA of 40.
I am guessing it was fairly close to those numbers prior to the 301. after three months, a lot should have moved to you. I looked at a cache of the new site and it goes to 2010 so I am assuming it was up when you did redirect and had been up for a while. The DA of the new site is 38 and home page PA is 39 so I would have expected that from a site up a couple of years. You don't appear to be getting much help from the redirect.
So, go back in and go url to url and make it work for you.
Let us know if there are other issues - great question as this happens to us all!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Where has the old seomoz crawl tool gone? I can't seem to find it
I'm looking for the (SEO)moz crawl tool - but can't find it. Where has it gone?
Moz Pro | | SearchMotion0 -
Why does Crawl Diagnostics report this as duplicate content?
Hi guys, we've been addressing a duplicate content problem on our site over the past few weeks. Lately, we've implemented rel canonical tags in various parts of our ecommerce store, over time, and observing the effects by both tracking changes in SEOMoz and Websmater tools. Although our duplicate content errors are definitely decreasing, I can't help but wonder why some URLs are still being flagged with duplicate content by our SEOmoz crawler. Here's an example, taken directly from our Crawl Diagnostics Report: URL with 4 Duplicate Content errors:
Moz Pro | | yacpro13
/safety-lights.html Duplicate content URLs:
/safety-lights.html ?cat=78&price=-100
/safety-lights.html?cat=78&dir=desc&order=position /safety-lights.html?cat=78 /safety-lights.html?manufacturer=514 What I don't understand, is all of the URLS with URL parameters have a rel canonical tag pointing to the 'real' URL
/safety-lights.html So why is SEOMoz crawler still flagging this as duplicate content?0 -
Is seomoz rogerbot only crawling the subdomains by links or as well by id?
I´m new at seomoz and just set up a first campaign. After the first crawling i got quite a few 404 errors due to deleted (spammy) forum threads. I was sure there are no links to these deleted threads so my question is weather the seomoz rogerbot is only crawling my subdomains by links or as well by ids (the forum thread ids are serially numbered from 1 to x). If the rogerbot crawls as well serially numbered ids do i have to be concerned by the 404 error on behalf of the googlebot as well?
Moz Pro | | sauspiel0 -
help with the inbound links side of seomoz
Hi Can somebody help with the inbound links tool of seomoz, can you point me in the direction of how it works and the best practices to get the most from it. I know i have a lot more inbound links to the site in my campaign but its shows for example 114 links but when i click on show more it only shows 3 . am i doing something wrong? Any help and advice from how people use it , what for and what is best practice thanks.
Moz Pro | | Bristolweb0 -
Crawl Diagnostics Summary
Sorry if I am not asking in the right place. On Crawl Diagnostics Summary it says this right..?? : "To get you started quickly Roger is crawling up to 250 pages on your site. You should see these results within two hours. The full crawl will complete within 7 days.". so it's passed a day and it still doesn't show nothing. It says "Processing Crawl Data for 358 pages" How much should i wait??
Moz Pro | | Dussk0 -
SEOMOZ Competitive Domain Analysis
Hi, I am just trying to get to the bottom of why I am seeing such wildly different figures in the SEOMOZ link analysis. The SEO company that we use is saying that we have around 331 external links to our website, when I check on Yahoo site explorer, it returns 264 external links, but in my SEOMOZ campaign manager, it says I have 4 external links? I fully appreciate that none of these methods are going to be wholly accurate, and are going to produce different results, but I can't understand why the SEOMOZ results are so vastly different. I also know for a fact there are a number of Followed links that I have added to a couple of websites, that Google has now indexed (as they appear in search results), but don't appear in the campaign manager results. If SEOMOZ use a different method to assess an external link, then no problem - my main concern here is that as a Newbie I am doing something wrong within the SEOMOZ tools, and don't want to take action based on results that may be incorrect because I have set things up incorrectly? Any help is greatly appreciated. Many thanks, Gareth
Moz Pro | | gdavies090319770 -
How do I get my crawl report?
I received a message that my crawl report is complete with a link - went to the link however - when I click on the icon that has the report name and the complete check mark nothing happens looked around can't find the results. Need to bid on this job so it would be helpful to know where to get it. Thanks for all you do. Wickey
Moz Pro | | Wickey0 -
What causes Crawl Diagnostics Processing Errors in seomoz campaign?
I'm getting the following error when seomoz tries to spider my site: First Crawl in Progress! Processing Issues for 671 pages Started: Apr. 23rd, 2011 Here is the robots.txt data from the site: Disallow ALL BOTS for image directories and JPEG files. User-agent: * Disallow: /stats/ Disallow: /images/ Disallow: /newspictures/ Disallow: /pdfs/ Disallow: /propbig/ Disallow: /propsmall/ Disallow: /*.jpg$ Any ideas on how to get around this would be appreciated 🙂
Moz Pro | | cmaddison0