Using Site Maps Correctly
-
Hello
I'm looking to submit a sitemap for a post driven site with over 5000 pages.
The site hasn't got a sitemap but it is indexed by google - will submitting a sitemap make a difference at this stage?
Also, most free sitemap tools only go up to 5000 pages, and I'm thinking I would try a sitemap using a free version of the tool before I buy one - If my site is 5500 pages but I only submit a sitemap for 5000 (I have no control of which pages get included in the sitemap) would this have a negative effect for the pages that didn't get included?
Thanks
-
Submitting a sitemap in Webmaster Console is always a good idea at any stage. If your website URLs are crawled and indexed in search engines than there will be no negative impact of it but in the longer run if you add more pages sitemap will defiantly a help.
If you are using CMS like WordPress, Joomla, Zencart or any other they all have extensions and plugins in their directory that will help you generate the sitemap of your current site and will add links as soon as you will add more pages.
Rest peter explains almost everything in detail like if you have URL issues and issues with crawling and indexing.
If you have a custom CMS, I think you should seriously consider the idea by Peter as this is something you need on regular basis anyways!
Hope this helps!
-
It's hard to tell without seeing your URL architecture.
First there are two specific terms and you never, never ever should forget them. They are - crawling and indexing. Once you prepare sitemap and submit there (or include in robots.txt) all bots get some map of your site and start crawling pages based on their crawling budget for your site. In crawling process they MAY find new pages that doesn't include in this map and will crawl them too. Again this is based on your crawling budget.
So when you submit sitemap - bot will get within seconds list of "non-crawled" 5000 pages and will start crawl them. Then he can find missed 500 pages and will crawl them too. Tricky is that when you update sitemap - he can detect quick changes there and start recrawling them again. But for missed 500 pages he can visit you again to check them for changes. And this will be also under your crawling budget. But if pages there isn't changed often - isn't big deal.
So you shouldn't hesitated about negative impact there. Only negative impact can happen if you have some serious URL architecture issues and messy URLs there. Then submitting partial sitemap can obfuscate this issues and some of your URLs to remain non-crawled.
Technically in SearchConsole you can see sitemap statistics like submitted and indexed. In perfect world numbers should be almost equal with little difference. But if you see huge difference between them - then you're in trouble. For example - on some site i have sitemap with submitted 44,950 pages and indexed of them was 29,643. This is pure example site crawling troubles or sitemap troubles. Because 1/3 of all pages isn't indexed at all.
PS: I forgot. You should use own CMS plugin for generating sitemap inside. Even if your CMS was custom made you should write (or hire someone) to create plugin inside. It's near 20-30 lines of write-here-your-favorite-language (PHP/Python/Perl/Ruby) and isn't big deal. This plugin will minimize crawling time from 3rd party sitemap generator tool because CMS already have all information inside and just need to be exported to XML.
-
It would definitely be better to submit a complete sitemap. If your site is built in Wordpress, Joomla, Magento, or many other standard CMS, it should have the ability to generate a full sitemap. Plugins like Yoast or Google Sitemaps help. Just depends on the site.
Otherwise you can probably get any pro SEO or agency to create a full 5500+ sitemap for you for $100 bucks or so.
PM me if you need more help.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Filter Tracking works fine at staging site but not on LIVE site why?
Hello Expert, For my ecommerce site I want to track filter url's like price range, size, width, color etc and fully filter url should display in google analytic. I have implemented filter tracking at staging server and it works perfectly but on LIVE site it not show me full filter url. Do you guys think any parameter which i have configured in search console affect this? Note - I have configured in this way - http://webmasters.stackexchange.com/questions/93008/how-to-track-a-product-filter-in-the-product-list-view-with-google-analytics My filter url's are given below. And in search console I have configure two parameters. 1) effect - Sort, Crawl - No urls 2) FT - effect- ( - ) , crawl - Let google bot decide. But as per me this parameter is for crawling should not affect tracking right? mysite.com?FP=0&filtSeq=Price&Sort=BS
Reporting & Analytics | | adamjack
mysite.com?FT=7581&filtSeq=Type&Sort=BS
mysite.com?FT=1042&filtSeq=Colour&Sort=BS In robot file nothing is block. In analytic it showing me url till mysite.com only where as in staging it shows me full filter url. Thanks!0 -
.com site referral traffic to ccTLDs
We have 7 international domains set up along with our main .com site. All of the ccTLds are showing their main referral traffic as coming from the .com site in GA, and most of those being from mobile. Each site is set up correctly with geo-targeting and hreflang tags. Has anyone experienced this before?
Reporting & Analytics | | ggpaul5620 -
Ecommerce site product link. How to handle a link that doesn't exist.
Suppose we have this product A, and we just have a single item for this. When the item is sold out we do not want to show it on the website saying "out of stock". Instead we would like to remove the product from out store which will now result in a url that doesn't exist. And google webmaster tool and Moz analytic will show them as page not found after they crawl over the site. Should i be generating a new sitemap.xml and update ? How do i handle those pages that don't exist anymore ? Thanks
Reporting & Analytics | | MindlessWizard0 -
Weird visitors to my site
Hi, I am in the process of disentangling myself from a dodgy SEO company. At some point they set up another GA account on my site without consulting me. I replaced the tracking code with my original account on my wordpress site, placing the tracking code on the dashboard. There is a box in the dashboard for you to do this. For some reason the account he created is still giving me analytics but from mostly one url :forum.topic55622342.darodar.com. It has marked it as a referral? When you click it it redirects to this site : http://activities.aliexpress.com/computers_channel.php?aff_platform=aaf&sk=vV3B2RJYB%3A&cpt=1421321021096&null There have been 218 visits from this "referral" in the last month and also 2 direct visits to a clients online gallery (i'm a photographer). I am guessing the code for this new account is still on the site somewhere? Funnily enough in the first month I was getting targeted by spam using my contact form and I was a bit perplexed as to why. We had to put captchas on the contact forms which I was loathe to do as its another step for a client to have to go through causing resistance. Has this link got something to do with it? I have recently disavowed a lot of toxic links he created, so maybe they had something to do with it? Best wishes. David.
Reporting & Analytics | | WallerD0 -
Can you tell MUV data on websites using MOZ?
I want to write reports on other websites and need to know MUV data on them
Reporting & Analytics | | WeAreVillage0 -
Best use of Keyword Rankings report
is there a guide for getting the most out of the keyword ranking tool? I currently have 250 or so keywords, some move up, some move down on a weekly basis but generally trend up. That's about as much info as I gain from it though. anyone got any good tips? thanks
Reporting & Analytics | | benj450 -
Bing Won't Index Site - Help!
For the past few weeks I’ve been trying to figure out why my client's site is not indexed on bing and yahoo search engines. My Google analytics is telling me I’m getting traffic (very little traffic) from Bing almost daily but Bing webmaster tools is telling me I’ve received no traffic and no pages have been indexed into Bing since the beginning of December. At once point I was showing ranking in Bing for only one keyword then all of a sudden none of my pages were being indexed and I now rank for nothing for that website. From Google I’m getting over 1200 visits per month. I have been doing everything I can to possibly find the culprit behind this issue. I feel like the issue could be a redirect problem. In webmaster tools on Bing I’ve used “Fetch as Bingbot” and every time I use it I get a Status of “Redirection limit reached.”. I also checked the CRAWL Information and it’s saying all the URL’s to the site are under 301 redirect. A month or so ago the site was completely revamped and the canonical URL was changed from non www to www. I have tried manually adding pages to be indexed multiple times and Bing will not index any of the sites pages. I have submitted the sitemap to Bing and I am now at a loss. I don’t know what’s going on and why I can’t get the site listed on Bing. Any suggestions would be greatly appreciated. Thanks,
Reporting & Analytics | | VITALBGS
Stephen0 -
Has anyone used pageinvasion as a plugin for wordpress?
Is this a good or bad for SEO and overall increase in business? Does it have negative effects on your relationship with google and other search engines?
Reporting & Analytics | | headeulogy0