Question about robots.txt
-
Solved!
-
Just a friendly reminder. Please don't delete your question after it's been answered. It's very likely that someone in the future could have the same question and they would have been able to find the answer if you hadn't deleted the question.
-
Consider deleting all of this:
Disallow: /&limit
Disallow: /?limit
Disallow: /&sort
Disallow: /?sort
Disallow: /?route=checkout/
Disallow: /?route=account/
Disallow: /?route=product/search
Disallow: /?route=affiliate/
Disallow: /?marca
Disallow: /&manufacturer
Disallow: /?manufacturer
Disallow: /?filter
Disallow: /&filter
Disallow: /?order
Disallow: /&order
Disallow: /?price
Disallow: /&price
Disallow: /?filter_tag
Disallow: /&filter_tag
Disallow: /?mode
Disallow: /&mode
Disallow: /?cat
Disallow: /&cat
Disallow: /?product_id
Disallow: /&product_id
Disallow: /?route=affiliate/
Disallow: /*?keywordThose rules are telling Google not to crawl domain.com/EVERYTHING(then the URL parameter). This could be where the issue stems from. If you're worried about URLs with these things ranking, consider implementing canonical tags instead to point to the proper pages
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Simple question: I wonder if I'm over internally linking?
Hi, What's you guys' policy on how much to internally link. I do it a lot - whenever it makes sense, but hold off if I just linked to the same page in the last paragraph, for instance. Would like to know your thoughts to see if I'm overdoing it. This is for Ecommerce blog posts, category descriptions, and product descriptions if that matters. Thanks.
On-Page Optimization | | BobGW0 -
Are there detrimental effects of having multiple robot tags
Hi All, I came across some pages on our site that have multiple robot tags, but they have the same directives. Two are identical while one is for Google only. I know there aren't any real benefits from having it set up this way, but are there any detrimental effects such as slowing down the bots crawling these pages? name="googlebot" content="index, follow, noodp"/> Thanks!
On-Page Optimization | | STP_SEO0 -
URL Question
This url looks bad: http://www.patrickmunoz.com/#!classes/c1vw1 And when you click around the page change doesn't actually occur, it's a fade into the next page. I think this is a major problem for rankings. Although pages are crawled: https://www.google.com/search?q=site%3Ahttp%3A%2F%2Fwww.patrickmunoz.com%2F&oq=site%3A&aqs=chrome.2.69i57j69i58j69i59l3j69i61.3548j0j7&sourceid=chrome&espv=210&es_sm=122&ie=UTF-8 When I search for a simple page - "patrick munoz FAQs" nothing comes up:
On-Page Optimization | | tylerfraser
https://www.google.com/search?q=site%3Ahttp%3A%2F%2Fwww.patrickmunoz.com%2F&oq=site%3A&aqs=chrome.2.69i57j69i58j69i59l3j69i61.3548j0j7&sourceid=chrome&espv=210&es_sm=122&ie=UTF-8#q=patrick+munoz+|+FAQs Do you think this is a bad url configuration? Thanks! Tyler0 -
Blog Question
Hello! I try to add as much fresh content to my site weekly, I do also have a blog that I update 3 times per week. Will google crawl frequently if I am updating info on my wordpress hosted blog when my blog name is www.blog.mywebsite.com? and the sites name is www.mywebsite.com? Will they treat them as separate domains? Thank you!!
On-Page Optimization | | TP_Marketing0 -
Drupal SEO Question
Still trying to get a handle on all this...a lot to figure out! My site is Drupal and a few weeks ago I had my developer install an SEO Module so I could add a meta description and keywords for my pages. I have done that for the majority of the landing pages and the product pages as well. Earlier I posted a question about the use of adding the meta description and keywords to each product. The answers were mixed and pointed out the problem of the duplicate content which I am trying to figure out how to fix. BUT...when I went through my On Page Analysis grading for my campaigns one of the reasons my grades were low were because of the use of meta keywords tag. That the search engines ignore them and competitors can use them against you. So now I am thoroughly confused. I should optimize each product page with the meta description and keywords or i shouldn't? Keywords should show up in the title tag and the content....anywhere else? Help! Thanks much, Shara
On-Page Optimization | | Confections0 -
Title tag questions
General title tag questions how important is it not to change your title tag, I own a prom dress site and I'm always changing them between prom and homecoming' Is it bad to have title tags that are only different by one word across thousands of pages on your site. I have thousands of dresses so each title tag only varies by the style # I have always had title tags that are for example black prom dresses, well i recently discovered that just black dresses is googled 10 times the amount so im debating changing them to black dresses so that the word dresses is closer to the front of the title tag, am I over reacting or is that a good idea or would it be bad to put black dresses, black prom dresses, black homecoming dresses I also put the year in almost every title tag 2012, is that bad, I ranked great for 2012 stuff but could it hurt my homepage domain rankings on major broader keywords
On-Page Optimization | | Dirty0 -
Photogallery and Robots.txt
Hey everyone SEOMOZ is telling us that there are to many onpage links on the following page: http://www.surfcampinportugal.com/photos-of-the-camp/ Should we stop it from being indexed via Robots.txt? best regards and thanks in advance... Simon
On-Page Optimization | | Rapturecamps0 -
The SEOmoz crawler is being blocked by robots.txt need help
SEO moz is showing me that the robot.txt is blocking content on my site
On-Page Optimization | | CGR-Creative0