Max Amout Of HTML Pages In A Folder
-
What's the maximum amount of html pages that one should put in a folder, to get the best SEO GoggleBot crawl? I'm aware that there's a limit of 10,000 on most servers, but was curious to know if a lesser amount of pages would be better, for crawling and indexing purposes. Also curious on peoples opinions on whether .jpg and .gif files should follow similiar rules.
-
Thanks for all the input. Google does seem to crawl everything these days, so I'm also in conclusion if the files fit, they'll get crawled. Sitemaps, internal links and optimized images are all a must.
-
For images, you want to make sure they're optimized for the web: small file sizes for easy download, but still a resolution that shows the image clearly. Your graphic designer and a good graphic design program (Photoshop, Gimp, etc.) should help with this.
-
Hi,
As Ray-pp said, there isn't an optimal number of pages that are going to serve you better.
However, if you want to help Google discover more about your site and pages of importances, look to create a good internal linking strategy. This doesn't mean that you should just add footer or sidebar links though - these are context links that talk about a different subject, along with a link to the appropriate page.
If you get this right, you can gain a lot in terms of Google understanding more about what you have to offer, and the links to primary pages can also lead to an increase in the SERPs for various phrases.
-Andy
-
AFAIK there is no efficient # of files to include in a folder directory for maximum crawl effectiveness. If you folder legitimately warrants 5k html pages in a directory, then Google will crawl all the pages. Make sure to create value-added pages with high quality content - Google will recognize them and crawl them as appropriate.
If you have the options, use your Google Webmaster Tools account to adjust crawl settings. Once your site is a specific size, Google will take-over crawl rate settings for you.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Website homepage must be first link from every page of sub folders which are related in content?
Hi, I have seen number of websites where they keep their website homepage as first link from every page of sub folders and even from sub domain pages some times. For example, giving website homepage as top navigational menu from every page in their blog. Will this helps in boosting the ranking of homepage if we link from sub folder or sub domain pages where they employ related content like blogs or help guides. Thanks
Intermediate & Advanced SEO | | vtmoz0 -
Outside Top 10 Even though - Higher Domain/Page Authority/Higher On Page Grade
Hi, Note: this is for Australian search results - for people in Perth.
Intermediate & Advanced SEO | | HeadStud
The website is: http://thedj.com.au I am trying to optimise for the keyword 'perth wedding dj', but also 'wedding dj perth' and for some reason my website isn't even in the top 10 results. Here is what's weird though: My on-page grade with the On-Page Grader for the keyword 'wedding DJ perth' is an 'A' for http://thedj.com.au (http://awesomescreenshot.com/0135135hca) When checking the Keyword Difficulty in the Google Australia search enginge for 'wedding DJ perth' - there are 4 results which have a lower domain authority than 15 (in fact one result has a domain authority of 1) - http://awesomescreenshot.com/03f5134zd1 http://thedj.com.au has a Domain Authority of 23/100 and a Page Authority of 34/100. (http://awesomescreenshot.com/0bb5134tb8) So seeing as the page has gotten an A for on-page optimisation for the keyword 'wedding DJ Perth' and has a higher domain authority then many results in the top 10... why isn't it in the Top 10?! Bonus Question:
Why is DJ Avi showing up at the top of search results (Local listing) depsite the fact that:
a) He has no website to link to
b) No reviews for his listing
c) No keywords that I can see (other than the fact that he's a DJ)
Screenshot: http://awesomescreenshot.com/05151349cb Meanwhile our Local Places - Thanks,
Kosta
http://www.headstudios.com.au0 -
Duplicate page title at bottom of page - ok, or bad?
Can I get you experts opinion? A few years ago, we customized our pages to repeat the page title at the bottom of the page. So the page title is in the breadcrumbs at the top, and then it's also at the bottom of the page under all the contents. Here is a sample page: bit.ly/1pYyrUl I attached a screen shot and highlighted the second occurence of the page title. Am worried that this might be keyword stuffing, or over optimizing? Thoughts or advice on this? Thank you so much! ron ZH8xQX6
Intermediate & Advanced SEO | | yatesandcojewelers0 -
Domain.com/keyword1.keyword2.html vs doamin.com/keyword1-keyword2.html
I was doing some research and saw this url structure in a website that was not ranking well and can't help but wonder was the url structure part of the problem as well it looks like this with a period between keywords. domain.com/keyword1.keyword2.html and was wondering if that is acceptable for search engines as opposed to the normal dashes like this expample ... domain.com/keyword1-keyword2-keyword3.html I have never noticed a period to separate words in a url before. Anyone have any experience with this ? Is this going to hurt possible rankings ? Thank you in advance, Joe
Intermediate & Advanced SEO | | jlane91 -
Is it possible we are being penalized for doorway pages?
www.trophycentral.comHi, we were hit very hard a few years ago by Panda and have been try to recover. We completely revamped our site (structure, canonical, duplicates, content, etc.) and had a couple of keywords recover. However, most traffic has not comeback and still cannot be found in the top 30 pages or so for major phrases. We have compared ourselves to major competitors in our industry and see very few differences. The only thing I see as a difference is that many of our products are in multiple sections. For example, we have all baseball awards (trophies, pins, medals) grouped together. We then have just baseball medals together and just baseball pins together. Is this something that could be causing us not to rank? I am asking because the phrases that are ranking are the ones that don't have multiple categories. We have no manual penalties, but now I am thinking this is what Google might consider a doorway page?As an experiment, I just noindexed all but one category for baseball and soccer to see what happens.Does this make sense? Has anyone else seen this?Thanks!
Intermediate & Advanced SEO | | trophycentraltrophiesandawards0 -
On-page optimization - Am I doing it well?
Hi Mozzers, I'm sitting here going through our site and optimizing all of our content.
Intermediate & Advanced SEO | | Travis-W
For the most part we've just written without correct keyword research, so the content lacks focus. Here is a page I would consider finished - http://www.consumerbase.com/international-mailing-lists.html I have our KWs in the: URL Title Tag Meta Description Bolded in Content Image Alt Attribute. If I optimize my other pages like this, will I be good?
It feels a tiny bit stuffed to me, but SEOmoz's on-page tool gives me glowing numbers. Thanks!0 -
Combining two pages into one
I've got two pages I'd like to combine into one. These two URLs: widgets.com/about_us.php & widgets.com/contact_us.php would become: widgets.com/about-us Two questions... 1. Is it okay to combine the content into one page and setup 301 redirects for both pages (without losing page rank)? 2. Do I keep the original pages in my sitemap.xml file or just take them out after I setup the redirect? Thanks in advance!
Intermediate & Advanced SEO | | seo-hunter0 -
How to Disallow Specific Folders and Sub Folders for Crawling?
Today, I have checked indexing for my website in Google. I found very interesting result over there. You can check that result by following result of Google. Google Search Result I aware about use of robots.txt file and can disallow images folder to solve this issue. But, It may block my images to get appear in Google image search. So, How can I fix this issue?
Intermediate & Advanced SEO | | CommercePundit0