Crawl Budget vs Canonical
-
Got a debate raging here and I figured I'd ask for opinions. We have our websites structured as
site/category/product
This is fine for URL keywords, etc. We also use this for breadcrumbs. The problem is that we have multiple categories into which a category fits. So "product" could also be at
site/cat1/product
site/cat2/product
site/cat3/productObviously this produces duplicate content. There's no reason why it couldn't live under 1 URL but it would take some time and effort to do so (time we don't necessarily have). As such, we're applying the canonical band-aid and calling it good. My problem is that I think this will still kill our crawl budget (this is not an insignificant number of pages we're talking about). In some cases the duplicate pages are bloating a site by 500%.
So what say you all? Do we just simply do canonical and call it good or do we need to take into account the crawl budget and actually remove the duplicate pages. Or am I totally off base and canonical solves the crawl budget issue as well?
-
agreed! we ran into the same problem with content (articles, etc). if you think of it in the same way as blog posts, they each have a unique URL, but with tags (i.e. categories) you are able to get them posted to the appropriate category landing pages.
have a somewhat related issue that i posted here
-
Another great way to go is to not put the category in the product URL. That was usually the best solution when I work on e-commerce sites.
-
Hi Highland,
I would defiantly work on making sure that your product only lives in one category. The canonical tag is a nice little band-aid but it still fix the root of the problem. I would suggest you can have it listed in many different categories but it only lives in one category at the product level. So for instance:
It's displayed here
site/cat1
site/cat2
site/cat3But it only displays product details at a url like this
site/category/product
I'm not a huge fan of having Google crawl 4 or 5 extra pages per product just to find a canonical tag when you could just spend the extra programming time to make it work correctly.
Casey
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Opencart vs. Wordpress/Woo
There are two issues facing me today. One is that my two e-commerce stores need updating after some 4 years, but I am seriously considering switching from Opencart to Wordpress/Woo. Opencart is a nightmare to work with at the best of times. Whenever I try to edit the footer of my current sites for instance nothing changes, the customisation of pages is sloppy and although the site works fine for perhaps the first 6 months, anytime after that it just slowly falls apart. Wordpress however features incredible customisation, is easy to edit the code but it lacks the backend functionality that Opencart is good at. Does anyone know the downsides of changing to Wordpress/Woo in respect to SEO?
Web Design | | moon-boots0 -
When rel canonical tag used, which page does Google considers for ranking and indexing? A/B test scenario!
Hi Moz community, We have redesigned our website and launched for A/B testing using canonical tags from old website to new website pages, so there will be no duplicate content issues and new website will be shown to the half of the website visitors successfully to calculate the metrics. However I wonder how actually Google considers it? Which pages Google will crawl and index to consider for ranking? Please share your views on this for better optimisation. Thanks
Web Design | | vtmoz0 -
Google text-only vs rendered (index and ranking)
Hello, can someone please help answer a question about missing elements from Google's text-only cached version.
Web Design | | cpawsgo
When using JavaScript to display an element which is initially styled with display:none, does Google index (and most importantly properly rank) the elements contents? Using Google's "cache:" prefix followed by our pages url we can see the rendered cached page. The contents of the element in question are viewable and you can read the information inside. However, if you click the "Text-only version" link on the top-right of Google’s cached page, the element is missing and cannot be seen. The reason for this is because the element is initially styled with display:none and then JavaScript is used to display the text once some logic is applied. Doing a long-tail Google search for a few sentences from inside the element does find the page in the results, but I am not certain that is it being cached and ranked optimally... would updating the logic so that all the contents are not made visible by JavaScript improve our ranking or can we assume that since Google does return the page in its results that everything is proper? Thank you!0 -
Canonical url with pagination
I would like to find out what is the standard approach for sections of the site with large number of records being displayed using pagination. They don't really contain the same content, but if title tag isn't changed it seem to process it as duplicate content where the parameter in the url indicating the next page is used. For the time being I've added ' : Page 1' etc. at the end of the title tag for each separate page with the results, but is there a better way of doing it? Should I use the canonical url here pointing to the main page before pagination shows up in the url?
Web Design | | coremediadesign0 -
How will engines deal with duplicate head elements e.g. title or canonicals?
Obviously duplicate content is never a good thing...on separate URL's. Question is, how will the engines deal with duplicate meta tags on the same page. Example Head Tag: <title>Example Title - #1</title> <title>Example Title - #2</title> My assumption is that Google (and others) will take the first instance of the tag, such that "Example Title - #1" and canonical = "http://www.example.com" would be considered for ranking purposes while the others are disregarded. My assumption is based on how SE's deal with duplicate links on a page. Is this a correct assumption? We're building a CMS-like service that will allow our SEO team to change head tag content on the fly. The easiest solution, from a dev perspective, is to simply place new/updated content above the preexisting elements. I'm trying to validate/invalidate the approach. Thanks in advance.
Web Design | | PCampolo0 -
Can google crawl text in jquery sliders?
We are redesigning our website and want to present a fair amount of text within jquery sliders. Will google crawl this text or is it treated the same way as actual script? Perhaps there is a way to just have the text as plain html but use jquery to display it?
Web Design | | Netboost0 -
Subdomain vs Sub Folder
Hi. This is my first question here guys and gals so please be gentle. We are creating a town based website that will hold news, events, articles and relevant local content. We are also creating a business directory that will be part of this. My question relates to an issue we have regarding how the directory will be treated as part of the "network". Although we aren't 100% sure on the domain name yet, we want the directory to be found as easily as the domain itself. Even better, if the two could be treated seperately but strengthen the overall theme for the domain we'd be in keyword heaven. Therefore in your opinion (hopefully with pro's and con's) which do you think will rank better in Google, Yahoo and Bing (or all three);
Web Design | | rufo
The directory seperate as a subdomain or as a folder within the main domain? So for example - providing we are including links between the two sections and the site is www.sitetown.com Which is better for the directory itself? http://www.sitetown.com/directory http://directory.sitetown.com or you're mad Steve, use a seperate domain altogether www.sitetowndirectory.com I hope you can help, obi-wan, you're my only hope..... PS - seomoz still rocks. Thanks
Steve0 -
Which Shopping Cart is best for SEO? Magento vs. X-Cart
Comparing X-Cart and Magento, which do you think is better for SEO and why? I am leaning towards Magento and wanted to get some opinions?
Web Design | | BlinkWeb0