Using Site Maps Correctly
-
Hello
I'm looking to submit a sitemap for a post driven site with over 5000 pages.
The site hasn't got a sitemap but it is indexed by google - will submitting a sitemap make a difference at this stage?
Also, most free sitemap tools only go up to 5000 pages, and I'm thinking I would try a sitemap using a free version of the tool before I buy one - If my site is 5500 pages but I only submit a sitemap for 5000 (I have no control of which pages get included in the sitemap) would this have a negative effect for the pages that didn't get included?
Thanks
-
Submitting a sitemap in Webmaster Console is always a good idea at any stage. If your website URLs are crawled and indexed in search engines than there will be no negative impact of it but in the longer run if you add more pages sitemap will defiantly a help.
If you are using CMS like WordPress, Joomla, Zencart or any other they all have extensions and plugins in their directory that will help you generate the sitemap of your current site and will add links as soon as you will add more pages.
Rest peter explains almost everything in detail like if you have URL issues and issues with crawling and indexing.
If you have a custom CMS, I think you should seriously consider the idea by Peter as this is something you need on regular basis anyways!
Hope this helps!
-
It's hard to tell without seeing your URL architecture.
First there are two specific terms and you never, never ever should forget them. They are - crawling and indexing. Once you prepare sitemap and submit there (or include in robots.txt) all bots get some map of your site and start crawling pages based on their crawling budget for your site. In crawling process they MAY find new pages that doesn't include in this map and will crawl them too. Again this is based on your crawling budget.
So when you submit sitemap - bot will get within seconds list of "non-crawled" 5000 pages and will start crawl them. Then he can find missed 500 pages and will crawl them too. Tricky is that when you update sitemap - he can detect quick changes there and start recrawling them again. But for missed 500 pages he can visit you again to check them for changes. And this will be also under your crawling budget. But if pages there isn't changed often - isn't big deal.
So you shouldn't hesitated about negative impact there. Only negative impact can happen if you have some serious URL architecture issues and messy URLs there. Then submitting partial sitemap can obfuscate this issues and some of your URLs to remain non-crawled.
Technically in SearchConsole you can see sitemap statistics like submitted and indexed. In perfect world numbers should be almost equal with little difference. But if you see huge difference between them - then you're in trouble. For example - on some site i have sitemap with submitted 44,950 pages and indexed of them was 29,643. This is pure example site crawling troubles or sitemap troubles. Because 1/3 of all pages isn't indexed at all.
PS: I forgot. You should use own CMS plugin for generating sitemap inside. Even if your CMS was custom made you should write (or hire someone) to create plugin inside. It's near 20-30 lines of write-here-your-favorite-language (PHP/Python/Perl/Ruby) and isn't big deal. This plugin will minimize crawling time from 3rd party sitemap generator tool because CMS already have all information inside and just need to be exported to XML.
-
It would definitely be better to submit a complete sitemap. If your site is built in Wordpress, Joomla, Magento, or many other standard CMS, it should have the ability to generate a full sitemap. Plugins like Yoast or Google Sitemaps help. Just depends on the site.
Otherwise you can probably get any pro SEO or agency to create a full 5500+ sitemap for you for $100 bucks or so. PM me if you need more help.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to configure multilingual site in google analytic? Currently showing in Referral Traffic why?
Hello All, Currently my Multilingual site is showing in referral traffic is it because I have not added hreflang tag on site? If yes and if I add the hreflang tag on all sites when where it will show in google analytic traffic from international sites? And what type of configuration required in analytic? Thanks!
Reporting & Analytics | | pragnesh96390 -
Is it possible to use Google Tag Manager to pass a user’s text input into a form field to Google analytics?
Hey Everyone, I finally figured out how to use auto event tracking with Google Tag Manager, but didn't get the data I wanted. I want to see what users are typing into the search field on my site (the URL structure of my site isn't set up properly to use GA's built-in site search tracking). So, I set up the form submit event tracking in Google Tag Manager and used the following as my event tracking parameters: Category: Search Action: Search Value When I test and look in Google Analytics I just see: "search" and "search value." I wanted to see the text that I searched on my site. Not just the Action and Category of the event.... Is what I'm trying to do even possible? Do I need to set up a different event tracking parameter? Thanks everyone!
Reporting & Analytics | | DaveGuyMan0 -
Why does a selection of sites I have written guest posts on not come up on my link analysis?
I have done a few guests posts on different sites and they are not coming up in my link analysis report.
Reporting & Analytics | | meteorelectrical
We created an info graphic on one particular site and this site isn't coming up on the link analysis report. Would there be a reason for this. I ran a check on the sites code and it doesnt contain "nofollow" as i originally thought this was the problem. Here is an example of our work on a site that isn't coming up on the analysis report. http://www.electriciansblog.co.uk/2013/10/energy-saving-using-led-lighting/ Thanks0 -
Using advance segments or primary dimensions?
Hi guys, I haven't played around with advance segments enough to know the answer to this, so looking for some help. What I want to track is a keyword through the whole analytics, so I created an advance segment for "KEYWORD". This segment is Include - Keyword - "KEYWORD". Now when I remove the advance segment and do the following: Traffic Sources - Organic - Primary Dimension - Keyword - "KEYWORD" The amounts of visits differ a lot. I'm guessing this has to do with sessions? Looking for a clarification on this.
Reporting & Analytics | | William.Lau0 -
Multi-Site Analytics Dashboards?
Anyone have recommendations on a good multi-site analytics dashboard? I am managing roughly 20 sites right now, and am looking for a dashboard that provides basic info like # of visitors, search traffic, etc. for a couple dozen sites at a glance.
Reporting & Analytics | | TakeshiYoung0 -
Using Google Analytics for tracking statistics of the HOMEpage
Hello, Lets say I have a website with the HOMEpage being http://www.page.com. There are various subpages e.g. http://www.page.com/page1, http://www.page.com/page2, http://www.page.com/page3 . Now when I use Google Analytics it gives me all kind of statistics for the subpages shown as /page1 /page2 /page3 ... (relative path). According to this pattern the HOMEpage should be just "/". Unfortunately I can't find "/" in any statistics. Is this normal? Thanks in advance
Reporting & Analytics | | guitarslinger0 -
Disclaimer about using Google Analytic?
When using GA on my Website do I need to have a disclaimer that 'We are using Google Analtyic to Track information'?
Reporting & Analytics | | daracreative0 -
Sort referring sites by visit change over time comparison in GA
I can't believe I've never done this before, so I'm going to assume that I previously must have figured it out via excel, but I'm hoping there's an easier way. So I want to compare the referring sites between April and May and see which have sent (specifically) less traffic. The problem with doing a comparison in GA is that it only sorts by the highest traffic for May, when actually I want to see the largest negative change (by number, not percentage) between April and May. Is there a way to do this via the dashboard or am I just going to have to play about in excel for 10 minutes?
Reporting & Analytics | | StalkerB0