Confused about repeated occurences of URL/essayorg/topic/ showing up as 404 errors in our site logs
-
Working on a Wordpress website, https://thedoctorwithin.comScanning the site’s 404 errors, I’m seeing a lot of searches for URL/essayorg/topic, coming from Bingbot, as well as other spiders (Google, OpensiteExlorer). We get at least 200 of these irrelevant requests per week. Seems like each topic that follows /essayorg/ is unique. Some include typos: /dissitation/Haven't done a verification to make sure the spiders are who they say they are, yet.Almost seems like there are many links ‘in the wild’ intended for Essay.Org that are being directed towards the site I’m working on.I've considered redirecting any requests for URL/essayorg/ to our sitemap… figuring that might encourage further spidering of actual site content.
- Is redirection to our sitemap xml file a good idea, or might doing so have unintended consequences?
- Interested in suggestions about why this might be occurring.
Thank you.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Robots.txt & meta noindex--site still shows up on Google Search
I have set up my robots.txt like this: User-agent: *
Technical SEO | | RoxBrock
Disallow: / and I have this meta tag in my on a Wordpress site, set up with SEO Yoast name="robots" content="noindex,follow"/> I did "Fetch as Google" on my Google Search Console My website is still showing up in the search results and it says this: "A description for this result is not available because of this site's robots.txt" This site has not shown up for years and now it is ranking above my site that I want to rank for this keyword. How do I get Google to ignore this site? This seems really weird and I'm confused how a site with little content, that has not been updated for years can rank higher than a site that is constantly updated and improved.1 -
My 404 page is returning a 404
Hi there, Moz has highlighted that my 404 page is returning a 404... Looking at webmaster tools within crawl errors, it's the same story. The only big change on the website is that we recently moved to https for the entire site, so all pages have a 301 to the corresponding https page, including the old 404 http page. I don't know if that makes any difference? Any help or advice on how I reasolve this will be much appreciated. Thanks, Stuart
Technical SEO | | Stuart260 -
How big is the problem: 404-errors as result of out of stock products?
We had a discussion about the importance of 404-errors as result of products which are out of stock. Of course this is not good, but what is the leverance in terms of importance: low-medium-high?
Technical SEO | | Digital-DMG0 -
"HTTP error: 404 not found" submitting YOAST SITEMAP
When I upload the YOAST site map to google webmaster i get "HTTP error: 404 not found" just for the portfolio tag and categories..For other things iI dont get any i kinf of errors Is it because i dont have any tags and categories of portfolio element? I have to say in my template I have the portfolio post option but im not using it. Tx
Technical SEO | | tourtravel0 -
302 error removing site from results
I have a client who had a screwy url structure based off of parameters and all. They hired a developer that added the keyword to the end of the url and set up 302 redirects to the new keyword included url. Since then the entire site has virtually gone missing in the results but it is not penalized. I put in a request with webmaster tools for reconsideration and they said there was no penalty. I only just found the 302 problem today and think this is probably the problem. Could this remove a site from the search results?
Technical SEO | | webfeatseo0 -
Increase in authorization permission errors error after site switch
We launched our new site 2 days ago , since site was down for 12 hours for maintenance, we saw google webmaster tool shows this error . Since then google hasnt crawled, its been 36 hours. Do we need to do anyting? We have close to a million page google crawled before and I am wondering if this will effect anything.
Technical SEO | | tpt.com0 -
Should I be redirecting a .gov.au domain to a .com/.com.au Marketing URL?
Hi everyone, First time post - what a great community! On a trial at the moment but will definitely be signing up to PRO. The situation I'm helping to promote an event in Australia (lets call it "myevent"). The event also goes by a nickname "myev" (which isn't searched as much as "myevent"). The event for the most part is promoted offline as www.myevent.com. As it currently stands (and appears to have been the case for a number of years), www.myevent.com and www.myevent.com.au 302 redirect to www.mylocation.gov.au/myev The event is pretty much number 1 for keywords around "my event" and "myev", despite little attention being paid to on page optimisation and issues around duplicate content which I intend to fix. The domain being indexed by Google is www.mylocation.gov.au/myev . Open site explorer stats Open Site explorer shows www.mylocation.gov.au/myev as having a strong domain authority, and hundreds of backlinks. www.myevent.com and www.myevent.com.au have domain authorities in the 20s and about 30-40 backlinks each. The conundrum I've had a chat with the IT guys running the site(s) and they mentioned they'll be switching the way the redirects work, so everything goes to www.myevent.com NOT the .gov.au URL. I've done a bit of reading on the forums and understand that 301s should be used in order to pass the juice/authority from the existing domain (in this case www.location.gov.au/mye). I understand not all of this will be passed. What I'm not sure about is which URL should be the final preferred destination. I may not have a choice - www.myevent.com has been the preferred URL for a long time - but want to know if this will affect the performance in search results if the .com.au isn't used as the final destination (even though we would be redirecting from it)? Any suggestions / ideas / help would be most appreciated. I hope this all makes sense - I'm relatively new to the whole SEO world! Best wishes and many thanks.
Technical SEO | | hergs0 -
How long does it take for customized Google Site Search to show results from pdf files?
The site in question is http://www.ejmh.eu I am pretty unsatisfied with the results I am getting from the Site Search provided by Google. We have over 160 pdf files in this subfolder: http://www.ejmh.eu/mellekletek The files are the digital versions of articles. When I search for content in those pdf files, Google does not show results. It does show results from older pages, dating back 1-2 years but it is certainly not showing anything from pdf files that I have just put up 3 weeks ago. My questions: If I place a Google Search on a site, does it not automatically display results from ALL the content in the root domain? Is there any correlation between how the Site Search is indexing the files and how Google is indexing the urls in general? Should I just wait and see whether site search performance improves or should I switch to another Search software like Zoom Search? It is vital to have a proper, high-quality search functioning on that site in the very near future. What are your experiences? Any tips are greatly appreciated.
Technical SEO | | Lauroca0