Test contet/pages indexed by search engines
-
During the web development stages of our Joomla CMS website, we have managed to get our site indexed for totally irrelevant test pages mainly to do with Joomla and some other equally irrelevant test content. How damaging is this to our domain from an SEO prospective and is there something we can do about it?
When we do a site:domain.com search we see hundreds of testpages with test/irrelevant meta tags etc.
-
Search engines regularly recrawl every website and will update their information based on changes you make to your site. It is a natural part of the internet. The "site under construction" information is not harmful, but in the future should be blocked from indexing.
-
Thankfully its only test urls that have been indexed by Google only.
However all 3 major engines have indexed our domain against "Site under construction" page with untitled/incomplete tags.
Is this harmful or will this be overwritten when we launch properly and get our site indexed?
-
When you begin developing a site, you should use the robots.txt file to block all search engine access to the site. This is one of the few times where a robots.txt file is very useful.
With respect to fixing the issue, it depends on whether the URLs will be used on the live site, how long it will be until your site launched, and whether unique URLs such as /testing were used or you are working with the same URLs which will exist on the live site.
If your site is still in testing and it will remain in testing for 30+ days, you could add the noindex tag sitewide. Once all the pages were removed from the index, you can then add the robots.txt file. Be careful not to adjust the robots.txt file prior to the pages being removed as the search engines wont be able to see the noindex tag.
You did not mention which search engine indexed your pages. If you are working with Google and the URLs will not exist on the live site, you could use the Google Removal Tool. This is really overkill and should not be necessary, but if the site owner is paranoid about the test pages causing damage to SEO you can take this approach. Any URL removed in this manner cannot be re-added to the index for 90 days.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate website got indexed: Caused rank drop?
Hi all, We have replica of our website with exact pages and content. That website got indexed by mistake and allowed for bots for more than 10 days. Our ranking dropped now and we moved from 2nd page to 5th page. But previously we had this happened and didn't hurt much. We got punished now? Thanks
Algorithm Updates | | vtmoz0 -
Have AMP pages on Chrome disappeared?
Hi, We amplified our site the other day and Google was quick to find our AMP pages. Today, we can´t seem to find any AMP pages on Chrome and it looks like we can´t find any other pages from sites like Wapo and so forth. They´re still visible on Safari. We are searching from Iceland. Is anyone experiencing this and if so, could you please share a link to the news? Thanks and have a nice day!
Algorithm Updates | | guidetoiceland0 -
Big hit taken on Google Search in Jan - Any Ideas?
Hello, I manage a news site that gets new items posted daily. We had had a pretty even keel with Google search and ranking for some time now only on the 9th Jan we took a massive drop and have no recovered except for one big spike on the 29th January. The only think we had done differently was not post as much over Christmas for about a week as people were on holiday but if this was the reason for it the posting is back to normal now and has been since the 6th Jan and nothing has recovered. The site is wjlondon.com - any ideas greatly appreciated. Thank you
Algorithm Updates | | luwhosjack0 -
Drop in Page Indexing, Small rise in Search Queries
Hello, I have a news based website so i am creating multiple new posts daily. I changed a lot of the site and got rid of old potentially duplicate content back in feb and had a sharp drop in pages indexed. I know this was because I removed a lot of pages though. However I still have a good 20,000 + pages on my site and my indexing has dropped a further three times since then. From 9,000 to 2,000 a coupe of months ago and then slowly down since April to just 133. It doesn't seem to have affected my search queries yet but surely will if it continues. I am really confused as to how this might happen & how to turn it around. We dont use any dodgy SEO tricks either.
Algorithm Updates | | luwhosjack0 -
AS we using the keyword related to our link but we are not listed in first page of Google search
AS we using the keyword related to our link but we are not listed in first page of Google search, but our competitors using the same keyword , they are listing in first page. how we can short this problem and get into first page on search
Algorithm Updates | | krisanantha0 -
How to search for popular press releases
I would like to research popular press releases in my industry. Ones that got picked up by many popular outlets, got a lot of coverage etc. Besides mindlessly searching the web for press releases, is there a better way? Almost looking for a service that ranks press releases in terms of effectiveness.
Algorithm Updates | | StreetwiseReports0 -
Does searching for brand names really affect search results?
Hey all, So I heard Rand Fishkin speak the other day at an event in Palo Alto and I tokk anyway an interesting little concept from the day. He mentioned that there might be some overall ranking benefits to having people search for your company's brand and hence arrive at your home page root URL. I was thinking and I'm going to put into place a little test to check this out, does anyone know if search results as a whole are positively affected by having a large number of searches for your company's brand name? I've referred to this in the past as "Branded Search" but perhaps there's another term... I'm interested in running a test whereby I actually remove our URL from the web almost entirely. From social media profile and potentially even from our business cards and instead replace this with "Google Us" or "Google: Junction Marketing". 25907011
Algorithm Updates | | blahblahblah20150 -
New visitors by search engines nowadays in USA and Germany
Hello, anyone have current data, how many new visitors in average drive nowadays search engines? My source of Peter Kent´s book SEO for Dummies from 2006, says around about 50% in the USA. I guess nowadays the percentage is much less, because of the social media? Have anyone data to split this in organic and paid search driven visits also? Do you know also how many new visitors in average drive nowadays social media like Facebook, Twitter & Co ? Thanks,
Algorithm Updates | | Braumueller0