Sitemap links
-
Hi, I´m running a sitemap using pro-sitemaps and I find several pages that shouldn´t be listed. How do I find how are these pages being generated? Can´t find the links the robot is following to get to those pages..
-
Screaming Frog is the best crawler out there imo. Have you talked to your hosting provider? They may be refusing the connection on their end.
-
Hi Thanks for your help! I keep getting "Connection Refused" status from the Screamin Frog..just couldn´t use it..I have tried the proxy, firewall and all recommended steps for solving this issue..
Is there a similar app?
Thanks!
-
Will try doing that thanks!
-
Have you considered using a tool like Screaming Frog SEO Spider?
It can crawl your site and it will list the inbound links to pages all neatly in a tab (exportable to Excel).
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Recovering from Sitemap Issues with Bing
Hi all, I recently took over SEO efforts for a large e-commerce site (I would prefer not to disclose). About a month ago, I began to notice a significant drop in traffic from Bing and uncovered in Bing Webmaster Tools that three different versions of the sitemap were submitted and Bing was crawling all three. I removed the two out of date sitemaps and re-submitted the up to date version. Since then, I have yet to see Bing traffic rebound and the amount of pages indexed by Bing is still dropping daily. During this time there has been no issue with traffic from Google. Currently I have 1.3 million pages indexed by Google while Bing has dropped to 715K (it was at 755K last week and was on par with Google several months ago). I know that no major changes have been made to the site in the past year so I can't point to anything other than the sitemap issue to explain this. If this is indeed the only issue, how long should I expect to wait for Bing to re-index the pages? In the interim I have been manually submitting important pages that aren't currently in the index. Any insights or suggestions would be very much appreciated!
Technical SEO | | tdawson090 -
Multiple Common Page Links
Hi everyone - I've taken over SEO for a site recently. In many cases, the reasons why something was done were not well documented. One of these is that on some pages, there are lists of selections. Each selection takes the user to a particular page. On the list page, there is often a link from an image, a name, and a couple of others. Each page often has 30 items with 4 links each. For some reason, the 4th of these internal links were no-followed. When I run this site through several different site evaluation tools, they are all troubled with the number of no-follow links on the site. (These instances from above add up to a 5 figure number). From a user perspective, I totally get why there is a link where each of these links exist. If I wanted to click on the image or the name or some other attribute, that totally makes sense. Its my understanding that Google / Bing are only going to consider the 1st instance. If this creates excessive links, wouldn't you want 3 of the 4 links in each set no-followed? If its only excessive unique links that really matter, then why would any be nofollowed.
Technical SEO | | APFM0 -
Referencing links in Articles and Blogs
Hi I am wondering if the <sup>tag in html is picked up by google as a reference point?</sup> I.e when you put a superscript in word it puts a small number next to your sentence. Then you have a list of reference at the end of the blog/article does google recognise this?
Technical SEO | | Cocoonfxmedia0 -
Will links be counted?
We are considering a redesign of our website and one of the options we are considering is to come up with something along the lines of http://www.tesco.com/, with rotating top offers. The question I am wondering is whether or not the links (ie. the blue links on the left side of the main graphic) will be visible to the spiders, and if not, whether there is a way to code it so they are?
Technical SEO | | simonukss0 -
Linking root domains and youtube
All of my competitors have high linking root domains from youtube and our isn't showing up although we have 1.5 million views to youtube. I tried adding our URL to the videos but it hasn't recognized as a linking root domain. What should I do?? There's a ton of SEO juice here I want to tap into! watch?v=GTXFRTY4CCA&list=UUOcfF9LAHKedNSyk-gk5xDw&index=28
Technical SEO | | tonymartin0 -
Links to Website Author
I'm a website developer, and in the past I have usually added a tiny backlink to the footer of my clients' websites like this: Website Design by MyCompanyName I understand that Google sees this as a low-quality backlink. However, I was wondering if such links can hurt my rankings. Does Penguin sees these links as spam? If so, should I add a rel="nofollow" to the links? Is there anything else I should change? I do not want to remove these links completely because they are good for marketing my business. I just want to minimize any negative SEO impact of the links. I appreciate your input. Thanks.
Technical SEO | | SiteWizard_LLC0 -
Internal linking to subdomains
Hi *, I have a main site called example.org, and a lot of user generated pages to foo.example.org / bar.example.org and so on. Most of those pages link back to example.org. In example.org I have a page that links to all subdomains. How can I optimize the pagerank of the list page? Should I add nofollow to subdomain sites to avoid passing link juice to those sites and keep normal linking from subdomain sites?
Technical SEO | | ngw0 -
Should I have a 'more' button for links?
I have a website that has a page for each town. rather than listing all the towns with a link to each, I want to show only the most popular towns and have a 'more' button that shows all of them when you click it. I know that the search engine can always see the full list of links and even though the visitor can't this doesn't go against Google guidelines because there is no deception involved, the more button is quite clear. However, my colleague is concerned that this is 'making life hard' for the search engines and so the pages are less likely to be indexed. I disagree. Is he right to worry about this??
Technical SEO | | mascotmike0