Firefox Add-On for crawl frequency??
-
Hi all,
a short one: is there a firefox add-on available, which lets you see the crawl frequency of your page(s)? Would be interesting to see if google bot comes around more lately...
There are some statistics in the webmaster tools, but I don't find them very attractive
I know there is something for Wordpress, but we don't use it...
I don't to put up an excel-sheet and check the cache-version for myself. And I would love to see how deep the crawler gets and which sites do not get crawled...
So, any existing add-ons or tools that are for free??
Thanx....
-
Hi,
Your best bet is Web master Tools or any other server side tool like the ones available for Word Press like you've mention. Depending on your server (unix or microsoft based) and/or your platform for the backend you can install / add additional features to get more info for this - google bot visits / crawl rates and formats.
There are no fireFox addons since the browser dosen't have access to your site / server in order to see when google bot is visiting what pages and how is it performing - as that 's a behind the scene type of process.
If you have access to your server you might try newrelic.com (I am not affiliate in any way with them - I just love the tool). However there are several others in the same spectrum that will give you more then the Google bot stats and data.
Hope it helps.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is a good crawl budget?
Hi Community! I am in the process of updating sitemaps and am trying to obtain a standard for what is considered "strong" crawl budget? Every documentation I've found includes how to make it better or what to watch out for. However, I'm looking for an amount to obtain for (ex: 60% of the sitemap has been crawled, 100%, etc.)
Technical SEO | | yaelslater1 -
Does Google add parameters to the URL parameters in webmaster tools/
I am seeing new parameters added (and sometimes removed) from the URL Parameter tool. Is there anything that would add parameters to the tool? Or does it have to be someone internally? FYI - They always have no date in the configured column, no effect set, and crawl is set to Let Google decide.
Technical SEO | | merch_zzounds0 -
Is there an easy tool to add all keywords I need to seomozpro?
In my website there are 3,300 products. www.theprinterdepo.com. I have about 200 keywords on the seomoz account. Every item has about 5 or more variations of keywords. examples: hp 1320 1320 printer hp 1320 refurbished 1320 hp printer hp laser 1320 and so on. The question is: Is there an easy way to add all the keywords related to my products and all its variations to the seomoz dashboard so I can more easily track them? If not, what do you suggest me to do?
Technical SEO | | levalencia10 -
Cloaking? Best Practices Crawling Content Behind Login Box
Hi- I'm helping out a client, who publishes sale information (fashion sales etc.) In order for the client to view the sale details (date, percentage off etc.) they need to register for the site. If I allow google bot to crawl the content, (identify the user agent) but serve up a registration light box to anyone who isn't google would this be considered cloaking? Does anyone know what the best practice for this is? Any help would be greatly appreciated. Thank you, Nopadon
Technical SEO | | nopadon0 -
Site maintenance and crawling
Hey all, Rarely, but sometimes we require to take down our site for server maintenance, upgrades or various other system/network reasons. More often than not these downtimes are avoidable and we can redirect or eliminate the client side downtime. We have a 'down for maintenance - be back soon' page that is client facing. ANd outages are often no more than an hour tops. My question is, if the site is crawled by Bing/Google at the time of site being down, what is the best way of ensuring the indexed links are not refreshed with this maintenance content? (ie: this is what the pages look like now, so this is what the SE will index). I was thinking that add a no crawl to the robots.txt for the period of downtime and remove it once back up, but will this potentially affect results as well?
Technical SEO | | Daylan1 -
Page crawling is only seeing a portion of the pages. Any Advice?
last couple of page crawls have returned 14 out of 35 pages. Is there any suggestions I can take.
Technical SEO | | cubetech0 -
E-Commerce Site Crawling Problem
Our website displays all of the products in our website If you attempt to visit a category or page that doesn't exist but conforms to our site url structure. Somehow google crawled these pages and indexed them, and they have TONS of duplicate content that hurt us. How do I deal with this problem?
Technical SEO | | 13375auc30 -
Two basic questions re. Crawl Diagnostic results
I'm a novice...I've just run my crawl diagnostics and I wonder how important is it to a) Have meta-descriptions on every page, b) Have all titles less than 70 characters? Thanks in advance. Dan.
Technical SEO | | danfk0