Moz Crawl Test: WordPress sites with and without /feed and /trackback entires?
-
I have multiple WP websites and on some of the websites, on my Moz Crawl test, I see an entry for every blog post but also entries for /feed and /trackback for that single blog post. For example,
www...com/someArticle
www....com/someArticle/feed
www...com/someArticle/trackback
1. Can anyone explain why the Crawl test is picking up the /feed and /trackback items? Is it simply because they are 301 redirects to the original post (www...com/someArticle)?
2. What setting(s) in WordPress are making this information appear? Or is it just that the site(s) that have the /feed and /trackback are displaying "normal" behavior for a WP site with a lot of trackbacks and feed entires?
3. Should /fee and /trackback, as well as /author be blocked in robots.txt?
Thanks in advance for your advice and input!
-
I have the same issue but instead of it redirecting to the parent post its just going to a 404 page.
-
So I solved the problem (or at least figured where it was coming from). On this particular site, under the comments area, there is a link for "trackback url" and a link for "comments rss feed". Naturally these are ../trackback and ../blog so that's why the crawl is picking them up. They are 301 redirected to the "parent" page so that's why they are not a duplicate content issue. Thank to everyone for their help!
-
1. If you check the source code of your blog posts, there must be some sort of link to the feeds - possibly even in the header. I'm not 100% on how the Moz crawler operates (if it only spiders <a>anchor links or if it spiders referenced links in the header - pretty sure the latter) - but either way that's how they're finding it, through some sort of link on the page.</a>
<a>You could try running a crawl with Screaming Frog SEO Spider and see if it also picks up the feed URLs and Screaming Frog will show you where it found the links as well.
2. Good question. Your theme may be displaying links to these things somewhere - the best way to find out is to crawl with Screaming Frog and it will show you which pages link to your feed and trackback URLs. Then if you don't need them, you can go into the editor and remove them from the code.
3. I agree with Thomas here, I would not block them with robots.txt - rather I would see if you can fix them at the source and remove the links if they are not needed.
-Dan</a>
-
Thanks, I'll check it out!
-
Hi, you should never block feeds they're really pretty beneficial to your site. Take a look at this from Joost it will explain it much better than I can
http://yoast.com/example-robots-txt-wordpress/
All the best sincerely, Thomas
-
Thank you.
When you say "TrackBacks are from people posting either identical or similar content to WordPress.com", what do you mean? I thought trackbacks were notifications of links back when someone links to your content?
And why does the codex recommend blocking feeds and trackbacks in robots.txt?
Thanks again!
-
the TrackBacks are from people posting either identical or similar content to WordPress.com I would follow up with that. unless that person is you.
No do not block a feed with robots.txt and do not block the TrackBacks use automatics Digital millennium act takedown if somebody is stealing your content.
Sincerely,
Thomas
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Looking for a MOZ pro to hire
Just got MOZ pro and need someone to look things over and give me a list of things to do first. When everyone claims to be an expert online. Any suggestions on who to hire or use?
Moz Pro | | Audio-Bible1 -
MOZ not accepting recent changes
We have made some changes on our website according to the SEO standards per Moz report and did some crawler tests but it seems that MOZ is not crawling our recent changes. It is showing the results according to the previous data. What can be the problem here?
Moz Pro | | torbett0 -
Meta Tag Descriptions not being found in Moz Crawls
Hey guys, I have been managing a few websites and have input them into Moz for crawl reports, etc. For a while I have noticed that we were getting a gratuitous amount of errors when it came to the number of missing meta tags. It was numbering in the 200's. The sites were in place before I got here and a lot of the older posts no one had even attempted to include tags, links of the page or anything. As they are all Wordpress Sites and they all already had the Yoast/Wordpress SEO plug-in installed on them, I decided I would go through each post and media file one at a time and update their meta tags via the plug in. I personally did this so I know that I added and saved each one, however the Moz crawl reports continue to show that we are missing roughly 200 meta tags. I've seen a huge drop off in 404 errors and stuff since I went through and double checked everything on the sites, however the meta tag errors persist. Is this the case that Moz is not recognizing the tags when it crawls because I used the Yoast Plugin? Or would you say that the plugin is the issue and I should find another way to add meta tags to the pages and posts on the site? My main concern is that if Moz is having issues crawling the sites, is Google also seeing the same thing? The URLS include:
Moz Pro | | MOZ.info
sundancevacationsblog.com
sundancevacationsnews.com
sundancevacationscharities.com Any help would be appreciated!0 -
Ignore Moz Notices?
Hi All, I read an old post (2011) about ignoring rel canonical notices from Moz. I wondered why they can be ignored? The thing is, it's showing a bunch of rel canonical errors for my pages but I know I have the tag right. At the risk of being offensive.. can the notices be ignored because the Moz data is erroneous? I'm not sure what to make of this. Thanks!
Moz Pro | | xvpn9020 -
Moz is officially demigod-approved!
So, I was reading Percy Jackson & the Olympians: The Battle of the Labyrinth tonight, and found a description of a graffiti tag in the Labyrinth that says "MOZ RULZ". What do you think? Is one of the Mozzers a demigod who has explored and survived the Labyrinth? 🙂 (See attachment for book quote.) F1WmT5r
Moz Pro | | AdamThompson4 -
Not all pages are being crawled
I am set up on the PRO plan, I was under the impression that it would crawl up to 10,000 pages. My site has just over 200 pages, but whenever I am crawled it only crawls 121 pages. Is this normal? It's hard to know how reliable my data is because a significant amount of pages are missing.
Moz Pro | | KristinHarding0 -
I need country specific ranking reports for our Google/Yahoo/Bing positions
Hi Guys, Using the ranking reports given here at SeoMoz I cannot select various countries as it seems to only provide US Google/Yahoo/Bing results. I need ranking reports specifically for Thailand, UK, States, Australia, India, Hong Kong. Does anyone know of any such tools? Many thanks in advance.
Moz Pro | | ewanTHH0 -
Crawl Diagnostics Summary
Is there a way to view the charts in the crawl diagnostics summary on a monthly view (or export the monthly figures)?
Moz Pro | | RikkiD220