Magento Robots & overly dynamic URL-s
-
How can i block all URL-s on a Magento store that have 2 or more dynamic parameters in it, since all the parameters have attribute name in it and not some uniform ID
Would something like:
Disallow: /?&*
work? Since the only thing that is constant throughout all the custom parameters is that they are separated with "&"
Thanks
-
Thanks, it worked
-
Your code should work, add it then test a dynamic url via GWT > Health > Blocked URLs
You can also set custom parameters to either be tracked or not tracked via GWT > Configuration > URL Parameters
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Trailing Slashes on URLs
Hi everyone I have a question on trailing slashes in URL. The crux of it is this: is having both: example.com/subdirectory/ and: example.com/subdirectory on all of your subdirectories considered duplicate content by Google - or in some other way really bad? We have done a heck a lot of research into this, and it would seem...no one knows for sure (it is easy to get lost in a sea of Webmaster tool forums from 2012). Google itself has both URLs for it's subdirectories (try https://www.google.co.uk/maps and https://www.google.co.uk/maps/) as does Moz; and yet there are some rumblings on the internet of people who think you must put a 'redirect' (although not really a redirect as it isn't a 301) in your htaccess file to one or the other (so for example.com/subdirectory/ would 'forward' to example.com/subdirectory); and this is what bbc.co.uk do. We tried putting this htaccess 'forward' in as an experiment, but I noticed our site then stopped being fully crawled by Google bot, so we reversed it. Can any one shed any light?
Technical SEO | | NickOrbital0 -
Exact Match Domain & Title Tag / URL
I currently own an exact match domain for my keyword. I have it set up with multiple pages and also a blog. The home page essentially serves as a hub and contains links to all the pages and the blog. My targeted keyword is on its own page and I made the title tag the same as my keyword. As an example the URL for my targeted post looks like this: benefitsofrunningshoes.com/benefits-of-running-shoes I have solid, non-spammy content and clean whitehat earned backlinks directing to that specific page. My concern right now is that the URL looks kinda spammy. The website has been live for about a week and the home page ranks well enough but my targeted page is no where to be found. (it does show up if I manually search via search command "site:benefitsofrunningshoes.com"). I'm wondering if it is acceptable to use the exact keyword in title tag / page url if it is also in the domain as an EMD? Should I change the title tag and leave the URL in? Or should I completely change the title tag and URL and 301 redirect to the new page? I appreciate any help!
Technical SEO | | Kusanagi170 -
Why are these URL's suddenly appearing in WMT?
One of our clients has suddenly experienced a sudden increase in crawl errors for smart phones overnight for pages which no longer exist and there are no links to these pages according to Google. There is no evidence as to why Google would suddenly start to crawl these pages as they have not existed for over 5 years, but it does come after a new site design has been put live. Pages do not appear to be in the index when a site search is used. There was a similar increase in crawl errors on desktop initially after the new site went live, but these quickly returned to normal. Mobile crawl errors only became apparent after this. There are some URL's showing which have no linking page detected so we don't know where these URL's are being found. WMT states "Googlebot couldn't crawl this URL because it points to a non-existent page". Those that do have a linking page are showing an internal page which also doesn't exist so it can't possibly link to any page. Any insight is appreciated. Andy and Mark at Click Consult.
Technical SEO | | ClickConsult0 -
Should I change the URL now?
Hi all, I have a client website that got hit in the latest algorithm update. It since appears that it had over 100 suspect links to it. I performed the Disavow procedure a few weeks ago via my Google Webmaster account, but have not received a message yet to say its been actioned. The majority of these suspect links go to one page. I am considering changing the base category (in Wordpress) to a different keyphrase and then submitting a new sitemap for indexing. This way there will be no actual link from a suspect website to a page on my website. Do you see what I mean? Will this help do you think? Thanks in advance.
Technical SEO | | BrandC0 -
Magento Redirect Loop
On my clients site I changed the URL structure for products. First they had a trailing slash: http://www.wellamy.com/market/food/oklahoma-relief-effort/healthy-snacks/caramel-apple-chewy-snack-bars/ I changed it to end in .html http://www.wellamy.com/market/food/oklahoma-relief-effort/healthy-snacks/caramel-apple-chewy-snack-bars.html Now rather than redirecting properly, it causes a redirect loop. Any idea how I can fix this?
Technical SEO | | Imajery0 -
Single URL not indexed
Hi everyone! Some days ago, I noticed that one of our URLs (http://www.access.de/karriereplanung/webinare) is no longer in the Google index. We never had any form of penalty, link warning etc. Our traffic by Google is constantly growing every month. This single page does not have an external link pointing to it - only internal links. The page has been indexed all the time. The HTTP status code is 200, there is no noindex or something in the code. I submitted the URL on GWMT to let Google send it to the index. It was crawled successfully by Google, sent to the index 5 days ago - nothing happened, still not indexed. Do you have any suggestions why this page is no longer indexed? It is well linked internally and one click away from the home page. There is still the PR of 5 showing, I always thought that pages with PR are indexed.......
Technical SEO | | accessKellyOCG0 -
Where to put 301 redirects in Magento?
I will be changing the URL structure in a magento store from "base/category/subcategory/product" to "base/product" which means i have to make over 1000 URL 301 redirects so our old links still work. Should i put the redirects in a .htaccess file so they stay intact no matter what or should i just put them in with all the other rewrites (in Magento 's URL rewrite manager)? Thanks
Technical SEO | | tilenkrivec0 -
Domainchange & Relaunch
Hi SEOMOZERS, We are planning to do a relaunch (new content, new design) + a domainname change (because it contains a keyword). Some of my team members think it is best to do the domain change already on the old-version to see if there might be some problems. Whereas I think it would be best to implement the domainchange on the new version because otherwise it can be suspected we did the domainchange only for SEO purposes. What are you guys thinking? Thanks Gregor
Technical SEO | | GregorHendrych0