Crawl Budget vs Canonical
-
Got a debate raging here and I figured I'd ask for opinions. We have our websites structured as
site/category/product
This is fine for URL keywords, etc. We also use this for breadcrumbs. The problem is that we have multiple categories into which a category fits. So "product" could also be at
site/cat1/product
site/cat2/product
site/cat3/productObviously this produces duplicate content. There's no reason why it couldn't live under 1 URL but it would take some time and effort to do so (time we don't necessarily have). As such, we're applying the canonical band-aid and calling it good. My problem is that I think this will still kill our crawl budget (this is not an insignificant number of pages we're talking about). In some cases the duplicate pages are bloating a site by 500%.
So what say you all? Do we just simply do canonical and call it good or do we need to take into account the crawl budget and actually remove the duplicate pages. Or am I totally off base and canonical solves the crawl budget issue as well?
-
agreed! we ran into the same problem with content (articles, etc). if you think of it in the same way as blog posts, they each have a unique URL, but with tags (i.e. categories) you are able to get them posted to the appropriate category landing pages.
have a somewhat related issue that i posted here
-
Another great way to go is to not put the category in the product URL. That was usually the best solution when I work on e-commerce sites.
-
Hi Highland,
I would defiantly work on making sure that your product only lives in one category. The canonical tag is a nice little band-aid but it still fix the root of the problem. I would suggest you can have it listed in many different categories but it only lives in one category at the product level. So for instance:
It's displayed here
site/cat1
site/cat2
site/cat3But it only displays product details at a url like this
site/category/product
I'm not a huge fan of having Google crawl 4 or 5 extra pages per product just to find a canonical tag when you could just spend the extra programming time to make it work correctly.
Casey
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I am Using <noscript>in All Webpage and google not Crawl my site automatically any solution</noscript>
| |
Web Design | | ahtisham2018
| | <noscript></span></td> </tr> <tr> <td class="line-number"> </td> <td class="line-content"><meta http-equiv="refresh" content="0;url=errorPages/content-blocked.jsp?reason=js"></td> </tr> <tr> <td class="line-number"> </td> <td class="line-content"><span class="html-tag"></noscript> | and Please tell me effect on seo or not1 -
AMP vs Responsive Design? Mobile SEO
Hello !! We are developing a new website with responsive design. As is recommended, the idea would be to have a unique site for mobile and desktop, with same content and same url for both devices, using responsive design to adapt the layout depending on the device. My doubt in here is about the AMP pages? If my website has responsive design, perfectly optimized for mobile do I need somehow AMP pages? As far as I understand, these amp pages would be useful if I had different pages for mobile, but this is not the case. Am I correct or am I missing something? Thanks for your help :
Web Design | | AutoEurope1 -
Specifying image dimensions for site speed vs. responsive
I'm working on improving site speed from an SEO perspective, and one bit of advice I see often is to specify the dimensions of the images you're using so the browser knows the size of the image it needs to download. However, I am wondering what impact this may have if the site is responsive? If you specify the large dimensions suited for a desktop browser, would you be forcing a mobile browser to use that sized image? Has anyone seen dramatic improvements in site speed using the <picture>tag for responsive images?</picture> Thanks! Jannette
Web Design | | JannetteP1 -
Help with 302 Temporary Redirect warning via MOZ crawl
Hi Guys, This is my first post so hopefully I'm using the forum correctly. MOZ crawl tells me that I have 35 pages with a temporary redirect The URL column displays 302 Found along with the http:// URL Redirection Location column shows the corresponding https:// URL This all seems pretty self explanatory. However, I’ve checked my .htaccess file and I can’t see any 302 references in it. I'm trying to figure out where the 302 redirects are from and how I can make them permanent Please can anyone help me out? My .htaccess looks like it needs a little tidy (there are 2 if blocks) <ifmodule mod_rewrite.c="">RewriteEngine On
Web Design | | ianalannash
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://www.mysite.com/$1 [R,L]</ifmodule> BEGIN WordPress <ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
RewriteCond %{HTTP_HOST} ^mysite.com$ [NC]
RewriteRule ^(.*)$ http://www.mysite.com/$1 [R=301,L]</ifmodule> END WordPress0 -
Drupal vs. Wordpress
I'm working with a site that is built completely in Drupal - including the blog. As we begin on a content strategy initiative the blog is vitally important but lacks core SEO functionality and is cumbersome for the content teams to work with as they have no technical background. While there will be more work required (moving the blog to a sub-domain or separate URL) to play nice with the broader drupal environment I believe it may be the better choice to move over to Wordpress and port over any existing content worth keeping. Has anyone encountered a similar scenario or have an opinion on either platform? Love to hear your thoughts both pros and cons. Thanks in advance.
Web Design | | JonClark150 -
What's the best way to structure original vs aggregated content
We're working on a news site that has a mix of news wires such as Reuters and original opinion articles. Currently the site is setup with /world /sports etc categories with the news wire content. Now we want to add the original opinion content. Would it be better to start a new top /Opinion category and then have sub-categories for each Opinion/world, Opinion/sports subject? Or would it be better to simply add an opinion sub-category under the existing news categories, ie /world/opinion? I know Google requests that original content be in a separate directory to be considered for inclusion in Google news. Which would be better for that? Regarding link building, if the opinion sub-categories were under the top news categories, would the link juice be passed more directly than if we had a separate Opinion top category?
Web Design | | ScottDavis0 -
Infinite Scrolling vs. Pagination on an eCommerce Site
My company is looking at replacing our ecommerce site's paginated browsing with a Javascript infinite scroll function for when customers view internal search results--and possibly when they browse product categories also. Because our internal linking structure isn't very robust, I'm concerned that removing the pagination will make it harder to get the individual product pages to rank in the SERPs. We have over 5,000 products, and most of them are internally linked to from the browsing results pages in the category structure: e.g. Blue Widgets, Widgets Under $250, etc. I'm not too worried about removing pagination from the internal search results pages, but I'm concerned that doing the same for these category pages will result in de-linking the thousands of product pages that show up later in the browsing results and therefore won't be crawlable as internal links by the Googlebot. Does anyone have any ideas on what to do here? I'm already arguing against the infinite scroll, but we're a fairly design-driven company and any ammunition or alternatives would really help. For example, would serving a different page to the Googlebot in this case be a dangerous form of cloaking? (If the only difference is the presence of the pagination links.) Or is there any way to make rel=next and rel=prev tags work with infinite scrolling?
Web Design | | DownPour0 -
Responsive Vs Mobile Sites
I know this is some cutting edge technology, but I think that this will be a very important topic in the coming months, as html5/css3 becomses more and more the standard, or at least standardized, I think the topic of this in relation to SEO will also arise much more. My question is simple, is it better to code a responsive site, or a completely mobile site for a small company with no special needs (mobile ordering, ecommerce, etc...) I obviously know the visuall differences, and, personally, I think respomsive websites look better. From an seo perspective, my big thing is for the resizing, for example, with WordPress, when you reach the tablet size you can set the sidebar to basically display:none, can that impact your website? I would really appreciate any feedback
Web Design | | ZacharyRussell0