Skip to content
NEW: Moz AI, Refreshed Interfaces & More API Data. Discover what's new at Moz!
Moz news 2 649533a

Use 'What Googlebot Sees' to Find Scrapers

B

This YouMoz entry was submitted by one of our community members. The author’s views are entirely their own (excluding an unlikely case of hypnosis) and may not reflect the views of Moz.

Table of Contents

B

Use 'What Googlebot Sees' to Find Scrapers

This YouMoz entry was submitted by one of our community members. The author’s views are entirely their own (excluding an unlikely case of hypnosis) and may not reflect the views of Moz.

Most of you are aware I recently took a position at Tribune as their in-house SEO manager.

One of the first things I did was get a current state of the union.  After working through the issues of verification for the dozens of sites (another post, perhaps), I realized that 'What Googlebot Sees' for a handful of the domains is 'Privacy Policy'.  Now, this is obviously a red flag for us to add a 'nofollow' tag on the link (that's perhaps another post also), but in addition to that, I found something else.

What's that?  What did I find?

Scrapers . . . lots of them.  You'd think they'd take the time to add a line of code to their bot to remove such a common piece of code as 'Privacy Policy', but . . . nope.

Take a peek at your Webmaster Central console under the 'What Google Sees' section.  Not only will it highlight what you should 'nofollow' but, if you dig a little deeper, you may find some scrapers.  What you do next is your decision.  Options? Take a legal stance, allow them to do it until you gain some more inbound links, reach out to them with an RSS feed (or other linking option), or do something a bit more blackhat, like making your 'Privacy Policy' type links an image with an alt tag for the targeted keyphrase.  ;-)  I'm not going to share what we're doing (though it is not the latter), as that would lead to proprietary information; besides, that type of decision is going to be pretty site-specific, anyway.

P.S. They turned the river green on St. Patrick's Day in Chicago.  Interesting, but not sure if it was worth standing in the cold for 45 minutes with my 4-year old son.  ;-)

Brent D. Payne

SEO Manager

Tribune Interactive

Back to Top

Snag your MozCon video bundle for even more SEO insights.

Read Next

That's a Wrap: The MozCon 2024 Day Two Recap

That's a Wrap: The MozCon 2024 Day Two Recap

Jun 05, 2024
Diving Into the Future of Digital Marketing: The MozCon 2024 Day One Recap

Diving Into the Future of Digital Marketing: The MozCon 2024 Day One Recap

Jun 04, 2024
Moz AI: Introducing Augmented Intelligence

Moz AI: Introducing Augmented Intelligence

Jun 03, 2024

Comments

Please keep your comments TAGFEE by following the community etiquette

Comments are closed. Got a burning question? Head to our Q&A section to start a new conversation.