Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Software/tool for Content Inventory

On-Page Optimization

2487

Locked

StickyRiceSEO last edited by

Is there any tool out there which can crawl a section of a website (or whole website) and gives a list of urls by subsections?

I have tried PowerMapper but it only gives visual map, not a list.

Thanks!
1 Reply Last reply
Reply Quote 0
hardy231088 last edited by

You can use Moz, Screaming Frog, Sort Site to get results.
1 Reply Last reply
Reply Quote 0
KevinBudzynski last edited by

Xenu mya work http://home.snafu.de/tilman/xenulink.html
1 Reply Last reply
Reply Quote 0
AbilityGroup last edited by

How about Screaming Frog? - http://www.screamingfrog.co.uk/seo-spider/
1 Reply Last reply
Reply Quote 0

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Content with changing URL and duplicate content

Hi everyone, I have a question regarding content (user reviews), that are changing URL all the time. We get a lot of reviews from users that have been dining at our partner restaurants, which get posted on our site under (new) “reviews”. My worry however is that the URL for these reviews is changing all the time. The reason for this is that they start on page 1, and then get pushed down to page 2, and so on when new reviews come in. http://www.r2n.dk/restaurant-anmeldelser I’m guessing that this could cause for serious indexing problems? I can see in google that some reviews are indexed multiple times with different URLs, and some are not indexed at all. We further more have the specific reviews under each restaurant profile. I’m not sure if this could be considered duplicate content? Maybe we should tell google not to index the “new reviews section” by using robots.txt. We don’t get much traffic on these URLs anyways, and all reviews are still under each restaurant-profile. Or maybe the canonical tag can be used? I look forward to your input. Cheers, Christian
On-Page Optimization | | Christian_T

2
Duplicate content shown in Google webmaster tools for 301 redirected URLs.

Why does Google webmaster tools shows 5 URLs that have been 301 redirected as having duplicate meta descriptions?
On-Page Optimization | | Madlena

0
Duplicate content with mine other websites

Hi,
I have a primary website www.vietnamvisacorp.com and beside that I also have some secondary websites with have same contents with primary website. This lead to duplicate content errors. What the best solution for this issue? Please help! p/s: I am a webmaster of all websites duplicate content Thank you!
On-Page Optimization | | JohnHuynh

0
Duplicated Content Column in excel

I'd like to see all duplicated content URLs in excel. But when I do the export to csv, and then use text to columns, I end up with an empty duplicated content column. The URLs should be in column AF in excel, but this column is empty. Can somebody help me on this?
On-Page Optimization | | jdclerck

0
Static content VS Dynamic changing content

We have collected a lot of reviews and we want to use them on our Categories pages. We are going to be updating the top 6 reviews per categories every 4 days. There will be another page to see all of the reviews. Is there any advantage to have the reviews static for 1 or 2 weeks vs. having unique new ones pulled from the data base every time the page is refreshed? We know there is an advantage if we keep them on the page forever with long tail; however, we have created a new page with all of the reviews they can go to.
On-Page Optimization | | DoRM

0
Geo-targeted content and SEO?

I am wondering, what effect does geo-targeted "cookie cutter" content have on SEO. For example, one might have a list of "Top US Comedians", which appears as "Top UK Comedians" for users from the United Kingdom. The data would be populated with information from a database in both cases, but would be completely different for each region, with the exception of a few words. Is this essentially giving Google's (US-based) crawler different content to users? I know that plenty of sites do it, but is it legitimate? Would it be better to redirect to a unique page, based on location, rather than change the content of one static page? I know what the logical SEO answer is here, but even some of the big players use the "wrong" tactic. I am very interested to hear your thoughts.
On-Page Optimization | | HalogenDigital

0
Checking Duplicate Content

Hi there, We are migrating to a new website, which we are writing lots of new content for the new website. The new website is hosted on a development site which is password protected and so on so that it cannot be indexed. What i would like to know is, how do i check for duplicate content issues out there on the world wide web with the dev site being password protected? Hope this makes sense. Kind Regards,
On-Page Optimization | | Paul78

0
Duplicate Content Question

On the home page of my site I have a read more link that takes you to a different URL with basically the same content, just more of it. Home Page: http://www.opwdecks.com/ Read More Link on Home Page: http://www.opwdecks.com/deckmaintain.htm I think this may be affecting my seo. Any suggestions on what I should do about this? Should I add a canonical to the home page and/or on the other page? Both pages are indexed by google. Thanks for any help or tips.
On-Page Optimization | | opwdecks

0