Tool that can retrieve mysite URL's
-
Hi,
Tool that can retrieve mysite URL's
I am not talking about href,open explorer, Majestic etc
I have a list of 1000 site URL's where my site name is mentioned. I want to get the exact URL of my site next to the URL i want to query with
Example
http://moz.com/community is the URL i have and if this page has mysite name then i need to get the complete URL captured.
Any software or tool that can do this? I used one for sure which got me this info but now i don't remember it
Thanks
-
Or a crawl test with moz pro tools
-
Yes, I forgot that he already had the list of 1000 sites. Xenu link sleuth would be another option--it's free.
-
That would show what's indexed (which is most) but not all pages
-
Do a google search for your "yourdomain.com" and then use a scraper tool to put the results into a google doc. Here's Seer Interactives tool: http://www.seerinteractive.com/blog/google-scraper-in-google-docs-update
-
Screaming frog SEO spider tool should be able to help you with this. However to crawl more than its' 500 URL limit, you will need to purchase a licence key.
http://www.screamingfrog.co.uk/seo-spider/
Good luck.
Regards,
Vahe
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Need Magento SEO expert for 301 clean up - any reco's?
My site is a total mess from a clean “crawling” perspective. We are still getting traffic and doing business, but I am afraid from an SEO perspective we are driving with the parking brake on. There a lot of 301’s and some of them are causing 404 errors. Below is an overview of my 5 year old magento site which was moved from a 5 year old xcart site (so there is a lot of old junk (url’s) in there). It needs cleaning up and I need a plan and seo / 301 help. Overview: Recently moved from http to https - not sure best practices were followed, but we had lots of crawl issues before this move. Analytics Top 100 Landing Pages = 82.7% of entrances Webmaster Tools 594 Pages Indexed 65 Not found errors - most involve 301’s - examples below Sitemap: 773 Submitted, 395 Indexed URL Parameters - 41 - I can’t tell if they are doing anything (helping or hurting) Moz Crawl Total Pages 3,454 324 Redirect Issues (258 Temp and 66 Chain) Magento 11,773 Redirects 5390 System 6383 Custom On July 15, 2017 I deleted 40 redirects from htaccess that a developer had put there that were causing problems. Blog We have a wordpress blog installed on Magento site. Years ago it was moved from a subdomain to a subdirectory.
Intermediate & Advanced SEO | | SammyT0 -
Old URL that has been 301'd for months appearing in SERPs
We created a more keyword friendly url with dashes instead of underscores in December. That new URL is in Google's Index and has a few links to it naturally. The previous version of the URL (with underscores) continues to rear it's ugly head in the SERPs, though when you click on it you are 301'd to the new url. The 301 is implemented correctly and checked out on sites such as http://www.redirect-checker.org/index.php. Has anyone else experienced such a thing? I understand that Google can use it's discretion on pages, title tags, canonicals, etc.... But I've never witnessed them continue to show an old url that has been 301'd to a new for months after discovery or randomly.
Intermediate & Advanced SEO | | seoaustin0 -
Someone asked me: What's the latest in SEO?
Hi, I'm wondering how others would respond to this question. "What's the latest in SEO?" Someone random asked me this on a plane that does not know much about digital marketing, but has someone else do for their business. I told them the google algortithm is constantly changing and it's always new, that there are about 500 changes a year (thought that was close to right) and then got down to some basic principals. I'm asking how you might answer as I could see someone asking me this within my organization as well. Thanks for any tips on a great answer or resources. Laura
Intermediate & Advanced SEO | | lauramrobinson321 -
How can I remove my old sites URL from showing up in Google?
Hi everyone. We have had a new site up for over a year now. When I search site:sqlsentry.net the old url still shows up and while those pages are redirected to .com I'd like to get the .net URL's out of google forever. What is the best way I can go about that?
Intermediate & Advanced SEO | | Sika220 -
Is there a downside of an image coming from the site's dotted quad and can it be seen as a duplicate?
Ok the question doesn't fully explain the issue. I just want some opinions on this. Here is the backstory. I have a client with a domain that has been around for a while and was doing well but with no backlinks. (Fairly low competition). For some reason they created mirrors of their site on different urls. Then their web designer built them a test site that was a copy of their site on the web designer's url and didn't bother to noindex it. Client's site dived, the web designer's site started ranking for their keywords. So we helped clean that up, and they hired a brand new web designer and redesigned the site. For some reason the dotted quad version of the site started showing up as a referer in GA. So one image on the site comes from that and not the site's url. So I ran a copyscape and site search and discovered the dotted quad version like 69.64.153.116 (not the actual address) was also being indexed by the search engine. To us this seems like a cut and dry duplicate content issue, but I'm having trouble finding much written on the subject. I raised the issue with the dev, and he reluctantly 301 the site to the official url. The second part of this is the web designer still has that one image on the site coming from the numerical version of the site and not the written url. Any thoughts if that has any negative SEO impact? My thought it isn't ideal, but it just looks like an external referral for pulling that one image. I'd love any thoughts or experience on a situation like this.
Intermediate & Advanced SEO | | BCutrer0 -
Can we retrieve all 404 pages of my site?
Hi, Can we retrieve all 404 pages of my site? is there any syntax i can use in Google search to list just pages that give 404? Tool/Site that can scan all pages in Google Index and give me this report. Thanks
Intermediate & Advanced SEO | | mtthompsons0 -
Google suddenly indexing and displaying URLs that haven't existed for years?
We recently noticed google is showing approx 23,000 indexed .jsp urls for our site. These are ancient pages that haven't existed in years and have long been 301 redirected to valid urls. I'm talking 6 years. Checking the serps the other day (and our current SEOMoz pro campaign), I see that a few of these urls are now replacing our correct ones in the serps for important, competitive phrases. What the heck is going on here? Is Google suddenly ignoring rewrite rules and redirects? Here's an example of the rewrite rules that we've used for 6+ years: RewriteRule ^(.*)/xref_interlux_antifoulingoutboards&keels.jsp$ $1/userportal/search_subCategory.do?categoryName=Bottom%20Paint&categoryId=35&refine=1&page=GRID [R=301] Now, this 'bottom paint' url has been incredibly stable in the serps for over a half decade. All of a sudden, a google search for 'bottom paint' (no quotes) brings up the jsp page at position 2-3. This is just one example of something very bizarre happening. Has anyone else had something similar happen lately? Thank You <colgroup><col width="64"></colgroup>
Intermediate & Advanced SEO | | jamestown
| RewriteRule ^(.*)/xref_interlux_antifoulingoutboards&keels.jsp$ $1/userportal/search_subCategory.do?categoryName=Bottom%20Paint&categoryId=35&refine=1&page=GRID [R=301] |0 -
Questions about Vittana.org's blogging contest and having bloggers use specific anchor text.
Hi All, Kenji Crosland here. I just joined vittana.org (yesterday!) to do some of the blogger outreach and content creation/link building. Although most of the links we've gotten in the past are branded links, we've decided to actively pursue anchor text links with specific keywords. If you check, you'll see that vittana has a relatively high domain authority. At the beginning of next week we'll be conducting a blogging contest with A-list celebrity tech bloggers. I don't think we'll have time to contact influencers in other areas for this contest unfortunately. When these A-list bloggers write their posts, we want them to have a link to this page: http://www.vittana.org/students To me, this seems a great opportunity to win on certain keywords we've discovered that should be easy to win and yet have a high volume of monthly searches. These are 5 word plus keywords that have over 300,000 searches per month. The students page, however, isn't optimized for those keywords. In the long run we want to win for the more difficult keyword "literacy". The word "literacy" is what we think will be a part of our new tagline: "Literacy is not enough". Because of time constraints, we won't be able to create landing pages to win for those "low hanging fruit" keywords in time for the blog contest. My question is: to what extent should we optimize the http://www.vittana.org/students page for the five word plus low hanging fruit keywords that we've discovered. I imagine if the content isn't relevant our clickthrough rates will suffer even if we do win for it (Altering our meta description is a possibility here) . Should we just try for the difficult keyword from the get go and come up with other ways to win for the low hanging fruit keywords? I'd love to hear your thoughts on this.
Intermediate & Advanced SEO | | vittana_seo0