Only 11 pages being crawled
-
Hi,
Can some one have a look and see why out of 400+ pages we only have 11 being crawled on here??
http://www.lifetimelegal.co.uk
Kind Regards
Elissa
-
When I search your site in Google it shows me 82 results against my query “site:domain.com” so it seems like Google is slowly crawling your site but you are right the rate of crawling pages is quite slow and to increase this here are few things that you should follow
- Strong Internal Linking
It is important to have a strong internal linking between the pages if you want Google and other search engine to crawl your page. Please check if you have a strong and powerful internal linking between the pages
- Xml Sitemap
The other thing that is missing on your site is xml sitemap. The idea is to add an xml sitemap and add it in Google webmaster tool and this will indicate Google to crawl different areas of your website with no hassle
Hope this will help!
-
Just looking around your site, the code seems ok - to a large extent.
But you are missing a sitemap (handy to point to all urls) and a robots.txt file
Also just clicking around i don't see 400 pages? is there a subdomain or folder you aren't linking to from the main site?
-
Hi Elisa,
It looks like it's pretty simple, when I crawl your Web site with Screaming Frog I only found 11 pages too. The reason is that the other pages aren't linked from the 11 pages found during a crawl. I would recommend to link to your other pages as well so these pages are found.
Hope this helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How does Google treat significant content changes to web pages and how should I flag them as such?
I have several pages (~30) that I have plans to overhaul. The URLs will be identical and the theme of the content will be the same (still talking about the same widgets, using the same language) but I will be adding a lot more useful information for users, specifically including things that I think will help with my fairly high bounce rate on these pages. I believe the changes will be significant enough for Google to notice, I was wondering if it goes "this is basically a new page now, I will treat it as such and rank accordingly" or does it go "well this content was rubbish last time I checked so it is probably still not great". My second question is, is there a way I can get Google to specifically crawl a page it already knows about with fresh eyes? I know in the Search Console I can ask Google to index new pages, and I've experimented with if I can ask it to crawl a page I know Google knows (it allows me to) but I couldn't see any evidence of it doing anything with that index. Some background The reason I'm doing this is because I noticed when these pages first ranked, they did very well (almost all first / second page for the terms I wanted). After about two weeks I've noticed them sliding down. It doesn't look like the competition is getting any better so my running theory is they ranked well to begin with because they are well linked internally and the content is good/relevant and one of the main things negatively impacting me (that google couldn't know at the time) is bounce rate.
Search Behavior | | tosbourn0 -
Long list or paginated pages
Hi peeps, I am just interested in this from a usability POV and to see what you would prefer to see when you are met with a page that has multiple options. Lets say that the page looks like a list of services, each clearly marked out in its own segment, but there are 50-60 options that match your requirements. Do you like to keep scrolling, or would you prefer to take what is there and then move on if you feel you want to dig deeper? Would you like to see a long list, of have the options loaded in as you get to them? -Andy
Search Behavior | | Andy.Drinkwater2 -
Is it better to find a page without the desired content, or not find the page?
Are there any studies that show which is best? If you find my page but not the specific thing you want on it, you may still find something of value. But, if you don't you may associate my site with poor results, which can be worse than finding what you want at a competitor site. IOW maybe it is best to have pages that ONLY and ALWAYS have the content desired. What do the studies suggest? I'm asking because I have content that maybe 1/3 of the time exists and 2/3 of the time doesn't...think 'out of stock' products. So, I'm wondering if I should look into removing the page from being indexed during the 2/3 or should keep it. If I remove it then my concern is whether I lose the history/age factor that I've read Google finds important for credibility. Your thoughts?
Search Behavior | | friendoffood0 -
Is putting/removing Adsense ads on the site affects crawling for seo?
Because I noticed when we totally remove Adsense ads on our site, the pages crawled per day on the google webmastertools suddenly dropped into noticable amount and we tested again to turn it on for a singe day, it jumped up again. So if Adsense affects crawling then, if we have adsense ad on all pages of the site the more chances it gets crawled by the bot? WS76R8w
Search Behavior | | CruiseControl0 -
Is a Shorter Page Title Better?
Is there any evidence that SEs give a greater weighting to keyword phrases in the page title if there are less characters? For example: 1. "Buy Silver Bullion" 2. "Buy Silver Bullion Coins Bars Rounds Easily Privately Securely" The key phrase I am trying to optimize for is "Buy Silver Bullion." To my knowledge, current practices would say the 2nd phrase is better optimized since it contains more keywords and it has a few USPs. But is there any evidence that the 1st example would be higher ranked in google for the phrase "Buy Silver Bullion" because it is more focused than the 2nd?
Search Behavior | | nwright0 -
Hi guys.. post-penguin my website coming in and out of serps every other day... what reason for that? ie. #11 --> #300+
We had one of the pre-penguin "unnatural links " messages in WMT - then Penguin hit April 24th/25th.. wee then set to work on Link profile... since 19th May we have had numerous search phrases come 'back' into rank pre-penguin for a day ie.. #5 - #10 and then drop the next.. ie 300+ this has been happening across a wide range of key phrases.... in fact it may well be that the key phrases are added to the serps 'briefly' or for a set amount of time... but our ranking checker checks daily so while it appears to be every other day in and out there may be another pattern... but the question is What is Google doing here and Why? any suggestions?
Search Behavior | | Geminineil1 -
How to optimize a Stock Symbol page?
Hi, My website is for stock picking. The keywords in each stock page are stock symbol. Now the stock symbol rank of my website in google is around 30-50. I use On-Page Keyword Optimization tool on seomoz to optimize those page. Now all stock symbol pages are Grade A in on-page report card. Is there other suggestions to increase stock symbol page ranking? Thanks in advance! -Don
Search Behavior | | dodoflying0 -
How to get more page impressions?
I'm wondering about one of our web-projects. There's a lot of good interesting content but the statistics of page impressions don't make me very happy. Each user is visiting just 1,5 sites per visit. That's really not much. We have other (similar) projects where this problem does not exist, where user are visiting a lot more sites per visit. I have no idea what could be the reason for it. Do you know / use some tricks to get more page impressions? Thank you, Sally
Search Behavior | | SallyO0