Find archived sitemap of a website that no longer exists
-
I am trying to figure out the site structure of a website and the urls of all the pages. Normally this would be easy but a couple of months ago the website went down and I don't think it will ever come back. Any help would be appreciated.
-
Use the internet archive (wayback machine) which effectdigital mentioned above, to find the /robots.txt file from the desired date. In that file you should find the referenced sitemap file (assuming the site properly included its sitemap reference in its robot.txt file). Then you can use the same process to request the sitemap file which was referenced in the robots.txt file.
-
Hi Effect,
Does your second link automatically provider the sitemaps available, or does the user still need to "know" or be able to guess where they might be e.g /sitemap.xml?
Nick
-
You can use this site to see legacy site-maps for some websites (though they may be partial or incomplete):
For example, check these sitemap results:
For smaller sites, the results are much easier to look at.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I am confused/frustrated/surprised how bad my website is doing on google ranking
Hello, I am confused/frustrated/surprised how bad my website (flyhy.co) is doing on google ranking and I have no clue why even though I have been doing my homework regarding SEO. Just a bit of background, I have created a new website about 6 months ago for the paragliding community, the primary goal is to provide a platform for people to publish their ads (osclass), but also to provide some interesting reviews and tools to help paragliders chose their wing. We have been putting a lot of effort to provide a nice user experience and tio build the tools mentioned above. Our main channel to connect with the community is Facebook, and we have been quite active there. I have looked at many SEO articles and I made sure the website provides a good UX, the URLs are SEO friendly, good meta data, etc. Also have been using the google search console and analytics to monitor all of this. But here is the thing, all these does not seem to change anything in our ranking for important keywords such as "paraglider for sale", "paragliding equipment", etc. We seem to only rank (looking at Google’s keyword tool) for very specific wing model names that people have mentioned in their ads. I have ran out of ideas on how to improve our SEO !!!!!! I know the website is only 6 months old, but by now we should get some results. As an example, I will mention one our main website competitors: www.paraglidingequipment.org. OK the URL is pretty obvious and this website ranks in page #1 for "paragliding equipment" (but also for "paraglider for sale" and other paragliding related key phrases). OK there is the URL (paraglidingequipment.org), but I thought nowadays google bots are smarter than just that. The website is 1 year old (so not really much older than us, and was ranking high anyway even 6 months ago). The website looks like it was clearly made by one person and then quickly just left it running, so no content has been added (except for people putting their ads), there is almost no activity on the Facebook account. I have run some test such as "pagespeed insights" and we both rank the same. On "seositecheckup.com", we are clearly better with more 10 points. Is there anyone out there who can tell me what is going on? Have I missed a very important aspect of SEO? Is our website somehow compromising the robots crawling (although I can see about 80 pages have been already indexed in google search console)? I know content is king, but in paraglidingequipment.org the only content I see are ads, and we have ads and other interesting (ie reviews and tools) for paragliders. To conclude, I am basically completely clueless of what to do to rank at least on the first couple of pages of google for the key phrases above. I need help. Hichem. PS: in Moz bar our score is non existing (PA=1,DA=10), on paraglidingequipment.org (PA=23,DA=15). So it looks that essentially we are not apparent on the web! PSS: We have also tried to build some backlinks on few important paragliding community websites.
Competitive Research | | hichemboudali0 -
Selling on eBay and Amazon, does it have a negative impact our your website?
_We sell on multiple platforms I.e own website, eBay, Amazon and have noticed over the last few years to hold a page 1 ranking on Google is becoming more and more difficult as the SERPs are saturated with the big brands. _ My question is, we've loaded all of our products and there lovely unique descriptions to eBay and Amazon, is there any chance that this content is helping eBay and Amazon rank (may be not by much!), but certainly not doing us any favours. As effectively why would you show our site in the search results for a product range, when all of the content/products already appear on eBay/Amazon which is several SERPs places higher? Is Google not inclined to think, "hey no need to show x site, as the content is already features on Amazon, why show it twice?" Any one have any thoughts?
Competitive Research | | bnknowles10 -
Why is my website not in the top 3 if our Moz statistics are better?
Hi, We've been using Moz, for our E-commerce sites, for some time now and are improving all of our statistics, be it on-page for specific keywords or general crawl errors, but I've not been seeing a lot of change in our rankings... (we've moved from place 7 to 6 on our main keyword). I'm thinking of looking at our backlinks, through Link Detox, after we finish our Moz optimisation to see if anything weird is going on with that. Are there any other areas I should be looking at? any other clues as to why we're not ranking higher than our competitors. Even if our Moz statistics are better and our Market Samurai values are similar? Thank you very much! Alexander
Competitive Research | | WebmasterAlex1 -
Free tools to find country of origin of backlinks/urls
Hey are there any free tools out there which can allow me to insert a large list of urls, and it determines the country of origin of the domain. I know the paid version of majestic does, but i was wondering if theres any free tools? Cheers, Chris
Competitive Research | | monster990 -
Twitter as a website's #2 ranked linked page?
A site I'm researching on open-site explorer has a #2 link with page authority of 52 and Domain authority of 97, and that link is the site's twitter page. No other sites I've researched have had their twitter page show up in it's link rankings like this, can someone explain?
Competitive Research | | TheSquareFoot0 -
Competitor Ranking High has 2 Domains, But Duplicate Website ?
I was using OSE and noticed all the backlinks to one of our competitors is there other domain name, which is the EXACT SAME website. You can enter both url's and they display the same content. They are not useing any canonical tags either. Why are they not penalized for duplicate content? And for using there own website for backlinks ? We try to do everything right, but still cannot beat them. Any thoughts on this?
Competitive Research | | hfranz0 -
Can i have chance to rank higher than official website in google local domains ?
Q : can i have chance to rank higher than official website in google local domains ? for example : rank higher than microsoft,kaspersky,nokia etc... in google italy or google germany or any other local domain for google
Competitive Research | | activeacts0 -
50,000 Links From The Same Website
Hey Mozers, I was conducting competitive research for a client of mine and I discovered one of their competitors has over 50,000 backlinks! After removing my heart from my stomach I realized that 99.9% of the links came from a single website that doesn't have nearly that many pages. Has anyone experienced this in the past? And if so what exactly is going on? C
Competitive Research | | calindaniel0