Magento Core_URL_Rewrite Problems
-
Hi Everyone,
We are currently caught between a rock and a hard place with Magento and are wondering if anyone else had similar problems and could share their advice.
Our Core_URL_Rewrite now containt 1.3 million records for an account that has 12000 products on 4 different store views. This has ballooned past the point that we are no longer able to reindex our URL Management.
The option that is being suggested to us is to truncate the table and start over, though this will essentially kill our SEO for those pages.(Which as there are duplicates, I can only imagine how much they are going to be penalized by it)
Would anyone have any advice other than truncating and starting over?
Any advice would be greatly appreciated.
Thanks!
-
Hi,
I found the exact problem you are facing with a solution on this link
http://magento.stackexchange.com/questions/17553/magento-core-url-rewrite-table-excessively-large
There are patch codes available on this link, however do read this reply on this page
Bugs in earlier (and possibly current) versions of Magento is one. Another is there's logic in this table that tries to track changes to the URL key value so that 301/302 rewrites are setup for old products. Because of this, and complicating things, truncating the table and regenerating may make existing URL rewrites go away, and this will have an unknown effect on your search engine listing (not necessity bad, just hard to predict).
My general advice to clients who ask is
-
Leave the giant growing table as is if you don't have a good handle on your URL/SEO situation
-
Until the table size starts being a problem (generating site maps, for example). When that happens, get a handle on your URL/SEO situation.
-
Once you have a handle on your URL/SEO situation, backup the table, then truncate the table and regenerate. Address any URL/SEO problems caused by the truncating.
-
Automate step 3
Trying to fix this on the Magento code level is admirable, but you'll be swimming upstream. Sometimes it's better to accept that "That's just Magento being Magento", and to solve the problem with and external process.
I hope this helps, if you have further questions, then post a response, I will be happy to answer.
Regards,
Vijay
-
-
I'm not sure the answers previously presented are related to the issues you're having. Having worked with Magento for a long time, this can be an issue that occurs over and over again.
To answer your initial question, truncating your core_url_rewrite table will remove all of these URLs, but it'll only delay the problem until it reoccurs again in the future (unless you've had a problem in the past which has been rectified). You're also correct in that any rewrites in the system previously there will disappear, so you'll probably end up with a lot of crawl issues appearing in Search Console.
Your best move would be to find out why you have so many URLs in there in the first place. Do you have a huge product catalog with multiple stores? Or is this something to do with an issue in your Magento version or some setup issues. The most common time this usually occurs is if two products get added to your site with the same URL Key. Every time the reindex process runs, your core_url_rewrite table will grow. You could check this by looking at the number of rows in the table, reindexing the site and if it grows further, then it's likely to be the problem. The quickest way to fix this is to ensure all URL key are unique.
There's also an article here about duplicate keys - https://firebearstudio.com/blog/magento-url-reindex-core_url_rewrite-duplicates-patch.html - this should hopefully clear the issue.
I hope this helps! If it doesn't solve the problem, then sending over a little more information around the number of stores, catalog site and the split between system generated URL rewrites and custom URL rewrites would be great so we can try to help further!
Thanks,
Lewis -
This is an issue to to set-up. When you set up multiple ecommerce websites on Magento as 'Stores', then all SKUs will load on other domains. if they were set-up as 'Websites' then this would alleviate the issue. However, with Stores you are able to share shopping carts (i.e. Add a product from website A and checkout on website B).
What I did was turn off the XML cron jobs and set-up cross-domain canonicals. Also make sure your session IDs (/?SID=) are working properly. Not sure if this solves the technical issues, but should help clear up dupe content.
-
Is it creating a new url for each option (size, color, etc) as well as what page it shows up on or other various sort orders (by price, by size, etc.) and session id's that you could exclude? Are you sure they are truly duplicates?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
HREF LANG: Different navigation/structure per country: is that a problem?
Hi all, One question about the href lang tag. Our webshop sells to 4 different countries (the Netherlands, Germany, Belgium & Spain). The navigation is a little bit different for these countries, depending on how popular certain product categories are in certain countries. So, for example: Netherlands --> Category A and B are in the top navigation
Intermediate & Advanced SEO | | AMAGARD
Germany --> Category B is a subcategory of product A. We want to implement the Hreflang tag, would it be a problem that the navigation/site structure (and therefore the URL structure for certain categories) are a bit different? So: The url for category B in the Netherlands is: https://www.website.com/nl/category-B/
The url for category B in Germany is: https://www.website.com/de/category-A/category-B/ Thanks in advance! Best!0 -
Big problems with site traffic
Hello! I have big problems with website promotion. It's been 7 months and the attendance on the site is 1-5 people a day. I do not understand the reason. Can you tell me what I'm doing wrong? Site: www.azartlist.com Many thanks.
Intermediate & Advanced SEO | | Bobic1 -
Help with facet URLs in Magento
Hi Guys, Wondering if I can get some technical help here... We have our site britishbraces.co.uk , built in Magento. As per eCommerce sites, we have paginated pages throughout. These have rel=next/prev implemented but not correctly ( as it is not in is it in ) - this fix is in process. Our canonicals are currently incorrect as far as I believe, as even when content is filtered, the canonical takes you back to the first page URL. For example, http://www.britishbraces.co.uk/braces/x-style.html?ajaxcatalog=true&brand=380&max=51.19&min=31.19 Canonical to... http://www.britishbraces.co.uk/braces/x-style.html Which I understand to be incorrect. As I want the coloured filtered pages to be indexed ( due to search volume for colour related queries ), but I don't want the price filtered pages to be indexed - I am unsure how to implement the solution? As I understand, because rel=next/prev implemented ( with no View All page ), the rel=canonical is not necessary as Google understands page 1 is the first page in the series. Therefore, once a user has filtered by colour, there should then be a canonical pointing to the coloured filter URL? ( e.g. /product/black ) But when a user filters by price, there should be noindex on those URLs ? Or can this be blocked in robots.txt prior? My head is a little confused here and I know we have an issue because our amount of indexed pages is increasing day by day but to no solution of the facet urls. Can anybody help - apologies in advance if I have confused the matter. Thanks
Intermediate & Advanced SEO | | HappyJackJr0 -
Keyword rich internal linking - problem?
Had an interesting situation today.. We write daily news articles on our site. In each article we link out to two sources that we are writing about (credible sources) and we do one or two internal links. For example.. 'Today McDonald's have announced that they are purchasing more blue widgets in order to increase their opportunity to appeal to a larger market.' So in that sentence you can see one outbound link and one inbound to blue widgets on our site. I got an email today from a large company who we have written an article about in the industry and they have asked me to remove the link to their site.. I actually asked them why and this was their response. 'We're concerned because of the number of keyword-rich internal links in the article, and are worried that being included alongside them might be misinterpreted by Google as an artificial link.' Fristly, do they really have anything to be worried about?.. but more importantly, with our internal linking, do we have anything to be worried about?.
Intermediate & Advanced SEO | | nick-name1230 -
Recommended e-commerce site search for Magento?
Does anyone have recommendations for any particular site searches for large e-commerce sites based on Magento? Some (hopeful) requirements: Possibility to segment product pages and blog content on results page Doesn't cause any major SEO or technical issues Understands semantic search Ability to filter results Ability to sort (e.g. by price, popularity, new in stock) It'd be really useful to see examples and know if there are any particular issues to be aware of. Thanks. 🙂
Intermediate & Advanced SEO | | Alex-Harford0 -
Www. & Non-www versions problem
I currently have both WWW and non-WWW versions of my site live on the web, and I want to be able to have all non-WWW to point to WWW.http://nile-cruises-4u.co.ukhttp://www.nile-cruises-4u.co.uk
Intermediate & Advanced SEO | | NileCruises
I want just the WWW version of the site live.I'm wondering the best way to resolve this?Is it normally dealt with in the .htaccess file and I think, looks like this but I'm not sure: <code>RewriteCond %{HTTP_HOST} !^www\. RewriteRule ^(.*)$ [http://www.](http://www.)%{HTTP_HOST}/$1 [R=301,L]</code>0 -
SEO & Magento Multistore - I have been asked if "duplicatiing" a magento stor using its "Multistore" functionality will cause both to be picked up as duplicate content, can anybody help?
Hello all. I have been asked what the consequences of using Magento's "multistore" functionality are if we were to duplicate our entire magento store and place it on a secondary domain... The simple answer which comes to my mind is that it will be a flagged as duplicate content. However, is this still the case if the site were placed in a different country? The original being the UK the copy being Ireland (both English speaking) How would Google.co.uk & Google.ie treat these stores? Hope this is clear... our site is http://www.tower-health.co.uk
Intermediate & Advanced SEO | | TowerHealth0 -
How do i get over my alt tage problems at a cateogry level?
At present at a category level, our site does not incorporate images specific to the category you are in and therefore we do not have appropriate alt tags to suffice SEO requirements.It only covers categories you are navigating too. e.g. http://www.towelsrus.co.uk/towels/catlist_fnct561.htm (no image placement available on page for that category, it only shows sub categories Does anyone have any suggestions how we get over this? How big a deal is it to not have image with appropriate keyword driven alt tag? Can you put more than 1 keyword phrase in a alt tag?
Intermediate & Advanced SEO | | Towelsrus0