How to find all 301 redirect for URL xyz.com/products (internal and external)?
-
This is what we are thinking:
- Get all URL of the xyz.com/products using XENU software.
- Search those URL on google (site;xyz.com url ) to find out if they are crawled by google, do the same on bing (as currently google shows 4k URL and bing 11k )
- Use opensiteexplorer (301 redirect ) and using (internal external) to get the desired result.
Is this the right approach? If not, what is the best way to find the correct result?
All suggestions are welcome.
-
You are welcome to work with the URLs in search results. I am unsure what numbers you are attempting to match.
-
Thanks @Ryan
I was referring https://www.google.com/#q=site:domain.com using site:domain.com , I know it doesnt give me all urls but isnt what we care about, at least matching numbers I mean?
-
You can check the redirects by uploading the LIST of URLs to Screaming Frog, which will then crawl the list and inform you of any header responses (301, 302, etc)
I am unclear on your second question. You previously stated the site involved is not a client and you do not have access to Webmaster Tools. What exactly are you asking or suggesting?
-
Thanks Ryan and everyone else, amazing answers So here is my understanding:
For Internal 301:
- Use Xenu or Screaming to scan url and create the list. I hope we can get a clean report from screaming.
For External:
-
Use Ose to find all backlinks and save them on excel
-
Use AHREFs to find all backlinks and save them
-
Use Majestic to find all backlinks and save them
-
Combine all url and remove duplicate(I guess manually we gotta do that)
Questions
a) How do we find out which one is 301 redirected beside checking each of them?
b) for backlinks we should check what google and bing crawled?
-
I agree - screaming frog is an awesome tool for finding the internal 301s. You can export a spreadsheet of all the pages that contain the 301s etc and so it makes creating a task list to work through (or pass on) pretty straight forward.
-
Your original question expressed a desire to locate "all" redirects for a given URL. It is highly unlikely to locate all such links without access to the link data in Google WMT and Bing WMT, along with the referrer data in GA.
The best you can do to find external URLs is to build a comprehensive backlink report using data from multiple providers (OSE, AHREFs, Majestic, etc). You should know from the start you will not cover all the links unless you are working with pages which have a low number of links.
-
Hi Ryan,
Thanks for your quick response.
Yes we heard about the screaming frog but never used it, will give it a try.
We do not have access to google/bing webmaster tool or analytics. We are more like a third party company working on this project.
Any other ideas?
-
I used to use Xenu until shortly after Dr Pete shared this blog (http://moz.com/blog/crawler-faceoff-xenu-vs-screaming-frog) which introduced me to Screaming Frog. Both will work, but you will find XENU is more like using DOS whereas Screaming Frog has a very nice interface.
For internal 301s, you can clearly crawl the site and export a list of all 301 redirects to the target page.
For external URLs, there is not a simple method. I suggest two tactics:
1. Examine your analytics for referring URLs
2. Examine your backlink reports for links to the page
You can then crawl the list of URLs and determine which pages are being redirected. With the above understood, the primary concern should be your internal URLs.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz Top Pages and Tool Bar Not Crawling Internal Pages and Links
Hello, We’re having two issues with our Moz tools and we’re not sure what’s causing them and whether they are related. The Moz Bar isn’t highlighting some of our internal links (including navigation links). The Top Pages Report in Open Site Explorer is only picking up the homepage and a couple error pages (none of the internal pages). The full Crawl Report is picking up everything though. Could a potential cause of both these issues be the Title attribute in some our links? – We use <a <="" span="">title="Example" href="link"></a> <a <="" span="">Or is this most likely from something else blocking the crawler from accessing our links/pages? Google Search Console does seem to be picking up the links in the navigation and everything is indexed/rendered correctly so we also didn’t know if this is something that could be issue. Any insight or help would be appreciated. Please let us know if there are any details we could provide that might help. Looking forward to hearing from all of you! Thank you in advance. Best,</a>
Moz Bar | | Ben-R0 -
Not sure where this url has come from
can anyone please let me know why this has happened on my site. I have just done a crawl test and it comes back with the following <colgroup><col width="576"></colgroup>
Moz Bar | | in2townpublicrelations
| http://howtodrinkless.com/web/20150709201150/http:/www.howtodrinkless.com/ |0 -
How to find a list of pages with missing H1 tags
An external SEO/PPC agency did an audit of our site a little while back and said that over 10% of our pages were missing an H1 tag. I am trying to find a way to gather a full list of these in order for our web company to fix. I downloaded the Crawl Test report thinking it would include info on tags in there but it doesn't seem to. Is there a different tool I can use that will get me this information?
Moz Bar | | Lepra0 -
External Links not showing in MOZ?
Hello:) I have a ton of external links on my site, but there not showing on MOZ? The reason i'm slightly concerned is because my top competitor has 80 and I show as only having 3? Does any one know why this would be? Also, my traffic on MOZ is a lot lower then my traffic report on Google Analytics? Is that normal, & which should I trust more? Here's my Site: Www.ThumannAgency.com Thank you in advance:)
Moz Bar | | MissThumann0 -
Sorry, but that URL is inaccessible?
Hi, I am trying to grade some pages and keywords using the "On-page grader" tool but for each URL that I try, the tool returns me a "Sorry, but that URL is inaccessible". The thing is that I have already used previously and without any problem some of these URLs. In fact, I have just realized that, while the same URL (for example: www.lacasadelaaldea.com) works in the "On-page optimization tool", it doesn't in the "On-page grader" right now. I have looked if someone could have experienced the same issue and I have found some other threads talking about it... so I have checked with my hosting provider that there is no firewall or any other thing causing this problem but they can't find anything. How do you make the call to the server? What could be happening? Thanks in advance, Juan
Moz Bar | | lcdla0 -
Ww.domain.com coming up with error
our domain is showing in moz with the following error in crawl reports Crawl Error We were unable to access your homepage, which prevented us from crawling the rest of your site. It is likely that other browsers as well as search engines may encounter this problem and abort their sessions. This could be a temporary outage, but we recommend making sure your network and server are working correctly. note that the url being displayed is ww.domain.com and not www.domain.com . we do not have a 301 in place, we have switched off wildcard forwarding from the server.. its acting as the url is a subdomain that is not working.. should i just ignore it?
Moz Bar | | Direct_Ram0 -
URL not returning a page successfully
I am trying to use the on page grader however it is not working with my website. The url is: http://www.britishhardwoods.co.uk/ Can anyone help?
Moz Bar | | British_Hardwoods0 -
Does Moz Pro generate similar keyword phrases in a list (preferably showing their difficulty %) or is it only one phrase at a time with no similar words/phrases suggested?
I just signed up for Moz Pro but the keyword research seems to only let you try one keyword phrase at a time. Is there a way for it to give related keywords along with their difficulty % info, etc. It is far too slow and inconvenient doing one at a time.
Moz Bar | | SavingSpotlight0