Need suggestion: Should the user profile link be disallowed in robots.txt
-
I maintain a myBB based forum here. The user profile links look something like this http://www.learnqtp.com/forums/User-Ankur
Now in my GWT, I can see many 404 errors for user profile links. This is primarily because we have tight control over spam and auto-profiles generated by bots. Either our moderators or our spam control software delete such spammy member profiles on a periodic basis but by then Google indexes those profiles.
I am wondering, would it be a good idea to disallow User profiles links using robots.txt?
Something like
Disallow: /forums/User-*
-
Maybe you should noindex user profile pages until they've been active on the site long enough that a) you're sure they're not a spam bot and b) they're actually active users, with some interesting information to share on their user page?
Like Shaliendra said, this is your call. It's really about your website goals, and what those user pages do for the site. Are you getting much organic traffic from user pages? If you aren't, it's probably easiest to just noindex all user pages. But if some profiles are bringing in organic traffic, I'd recommend that you add a noindex tag immediately, but remove it once someone has commented a few times.
Does that help?
-
I am only talking about user profile pages not the links inside them. I have already taken care of them.
Request others to please come up with suggestions.
-
If you are talking about user profile pages, it is your call.
If you are talking about links that users post in profile pages, use as signatures or use in threads, make sure to mark them nofollow.
Regards
-
Let me re-phrase my question a bit. Would it be a good idea to get the user profile links noindexed in the first place? or should we keep them indexed in Google?
-
Use meta="noindex" tag for the user profile pages. It is more effective.
Regards
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Need help with Meta tags generator
Hello, I am doing SEO for my business website. I am trying to generate Meta Tags here
On-Page Optimization | | webdropyyc
https://webdrop.ca/seo-tools/meta-tag-generator What should I enter for "Search engines should revisit this page after ". We are not going to change content on daily basis. 7 Days or 30 days are good enough?0 -
Optimizing shop content for desktop and mobile users
When arranging content on a shop category page I place a descriptive optimized opening paragraph of text above products. On desktop this shows both the opening text and the products above the fold (visible here https://www.scamblermusic.com/royalty-free-music-downloads/ - also shown on the screen grab below). The text may well be ignored by most visitors (who will likely be drawn straight to product images) but it still serves a purpose. dekstop.png When it comes to smaller mobile screens I have started to disable the opening paragraph of text (above the products) and instead place a copy of it below the products, (screen grab below). This keeps the optimized text on the page, but it means that mobile users instantly see products rather than having to scroll past text that they may see as inconvenient. mobile.png I'm conscious of the fact that Google indexes mobile content first, and it also doesn't like duplicate content. I therefore have three questions relating to this: Will moving the optimized text content below all the products to the bottom of the page devalue it (I understand important content should be as near to the top of page as possible)? Although the optimized paragraph of text only displays once on desktop (at the top of the page) and once on mobile (at the bottom of the page) it is actually visible twice in the source code - does this count as duplication, and could it therefore hurt the performance of the page in SERPs? If this practice does cause issues, is there an ideal way to optimize content on pages (especially shop category pages) that doesn't require mobile users to scroll through text before seeing products? Lastly, on topic optimized landing pages that feature product promotions such as this one - https://www.scamblermusic.com/royalty-free-music-downloads/music-licensing-scotland/ - I wonder if it is best to lead with an optimized text introduction above product images, or better to place the products right at the top of the page for immediate impact, then follow this with the content/article/blog post? Many thanks for any advice offered.
On-Page Optimization | | JCN-SBWD0 -
Internal Link Title and Anchor Text Question
So, I am kind of a newbie with all of this stuff, so please bear with me! If you have a chance to look into the domain it's www.noip.com I have checked the internal followed and 301 links. It seems like it is pulling in all of our navigation a funky way. all of those link have no data for the title and the anchor text is Home, Contact Us, Forgot Password?, ETC. Is this correct? I have tried to look into the way other websites are doing it and it seems wrong. I tried to talk to my developer, but he is not so sure. Do we simply need to make more copy on our site that has internal links within the copy with anchor text that is for keywords that I would like to be ranking for? Like Managed DNS? and then that page goes to the managed DNS page? Or what? Sorry if i am not being totally clear. Any help and advice is 110% welcomed!!! Natalie
On-Page Optimization | | vwnatalie0 -
Footer-links (navigation)
I want to put navigation links in my footer. I watched the (distilled) whiteboard friday video saying it can be a penalty. How are links in the footer different from links in the navigation. Why don't you get "hit" for putting the word "Real Estate Listings" in the navigation menu? It appears on every one of the 30k pages. I don't get why the links in the footer would have a negative effect if the links in the navigation don't. My example is a real estate website. When people are browsing properties (30k), it is convenient that at the end of the page there is a navigation of areas. I could put the navigation on the sidebar but that would be in essence, the same thing. A wordpress site would dynamically include that sidebar on every page.
On-Page Optimization | | JML11790 -
Blocked By Meta Robots
Hi I logged in the other day to find that over night I received 8347 notices saying certain pages are being kept out of the search engine indexes by meta-robots. I have not changed my robots.txt in years and I certainly didn't block Google from visiting those pages. Is this a fault on Roger Mozbot behalf? Or is there a bot preventing 8000+ pages being indexed? Is there a way to find out what meta-robot is doing this and where? And how I can get rid of it? I usually rank between #3 and #5 for the term 'sex toys' on google.com.au, but I now rank #7 to #9 so it would seem some of my pages/content is being blocked. My website is www.theloveshop.com THIS IS AN ADULT TOYS SITE. There is no porn videos or anything like that on it, but just in case you don't wish to look at sex toys or are around kids I thought I would mention it. Blake
On-Page Optimization | | wayne10 -
More than 100 internal links from a page
Hi, we have been developing our new site and improving the internal linking for 2 reasons, 1 to improve spidering and 2 to up sell more to customers. The error reports from SEOMoz are showing our biggest problem is too many internal links from 2000+ pages. How much of an impact does it have by having say 180 internal links compared to say 99 on a page? Our website has been moving up the SERPs so should i worry about it or should I ignore the warnings and continue with the menu system and internal linking we have in place already? Thanks
On-Page Optimization | | PottyScotty0 -
How do we handle sitemaps in robots.txt when multiple domains point to same physical location?
we have www.mysite.net, www.mysite.se, www.mysite.fi and so on. all of these domains point to the same physical location on our webserver, and we replace texts given back to client depending on which domain he/she requested. My problem is this: How do i configure sitemaps in robots.txt when robots.txt is used by multiple domains? If I for instance put the rows Sitemap: http://www.mysite.net/sitemapNet.xml
On-Page Optimization | | nordicnetproducts
Sitemap: http://www.mysite.net/sitemapSe.xml in robots.txt, would that result in some cross submission error?0 -
Best article about internal linking structure?
Hi! Could you please recommend me a good and deep article about best practises in internal linking structure? I need to rethink the structure of a big site (lucky me it's very hierarchical) and I would like to have a look at some great articles about this to consolidate some ideas and have some new ones. I've read some but I would like some recommendations 🙂 Some articles about information architecture would be appreciated as well! Thanks!
On-Page Optimization | | jorgediaz0