How to make google crawl our repository to make our site rank but make sure users dont go to our repository ?
-
We have a website that has links to documents related to various sectors. But the challenge is we do not have the documents on the website itself and they are linked to our document repository that has been blocked to google. We have put nofollow and noindex to the repository. Since Google can not read those documents, it has resulted in an impact in our SEO ranking. What would be the best way to make Google crawl the PDF documents in the repository at the same time make it invisible the "repo" not appear in the search engines. Would dofollow and noindex sequence work ?
-
Playing with indexation tags can be dangerous (same goes for robots.txt). Google should still be able to read the repo even if it is no-indexed, as long as you haven't also blocked the repo in robots.txt. Robots.txt is telling Google what to crawl, no-index is telling Google what it can or can-not put in its search results
Of course, if your docs were ranking because of PageRank passed from the repo, the no-index tag will kill the PageRank of the repo (and thus all the docs which it links to, as they are not being 'fed' any more). If a page is no-indexed, it's seen as unimportant for Google and the PageRank is often nullified. Although Google can crawl no-indexed URLs, they crawl them WAY slower as they're seen as really unimportant with no PageRank (at the bottom of the internet)
Why not just put all your PDF docs in a PDF sitemap ans submit to Google in Search Console:
https://stackoverflow.com/questions/1072880/should-i-list-pdfs-in-my-sitemap-file
This will let Google see them all. But if their parent is no-indexed with no PageRank, they may still not rank as well as before...
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Setting up a remote business with their Google Business Account
Hi all, I have a client who operates a remote business and I need to get her Google business/ brand account set up. She doesn't want it listed under her home address (for obvious reasons) but that is where her business is based out of. Apart from getting a PO box and listing her business under that for Google, does anybody know of any other options or best practices? Thanks!
SEO Learn Center | | Zx30 -
Should I issue a change of address for my demo site which has been indexed?
The web design company tasked with building a company website have a demo site which reflects the live site - as the site is constantly being developed and needs to be tested by numerous uses, removing the demo site is not an option. The demo site is a subdomain of the live site: demo.mysite.com The demo site has been indexed by Google as the web design company didn't add a robots.txt file disallowing all. The demo site was indexed 3 years before I started working at the company. My first action was to add a robots.txt file disallowing all, I then added a noindex and nofollow meta tag into the header of all files on the demo area - this has prevented any new pages from being indexed but the original pages of the site are still in Google index. Do you think I should add the demo subdomain (demo.mysite.com) into Google Webmaster and issue a change of address pointing to the actual website (www.mysite.com) or is this going to cause problems? All thoughts, opinions and recommendations are very much appreciated.
SEO Learn Center | | iam-sold0 -
Google Analytics setup tips and tricks?
I want to see (and possibly compile) other marketers tips and tricks when it came to setting up their Google Analytics. I feel like I sometimes get into a rut when I setup GA for clients and I want to make sure that I'm not getting rusty with it. Some of the things I'm interested in (but not limited to): How do you use dashboards? What funnels do you setup? How do you utilize goals? What are your favorite metrics to report to clients? What are the top items on your checklist when setting up GA? And many others I could include here... I'd be interested in seeing what other marketers include in their GA setup and reporting. These can be super simple or expert level tips and tricks. It's a personal curiosity I have so that I make sure I'm staying up to date with everything but if there is enough participation I will compile everything into a YouMoz post or something as a guide for us to use and share. So please share this question with others if you can. Thanks!
SEO Learn Center | | HashtagJeff2 -
I changed the {Site Name} of our domain by capitalizing a letter. 48 hours later our SEO rankings are bad. How long does it take to rebound our prior rankings. The {site name} was at the end of every page title.
I changed the {Site Name} of our domain slightly and capitalized a letter. The impact 48 hours later on SEO is really, really bad. How long does it take to recapture our prior rankings and should I change the site name back to the original? The {site name} was at the end of every page title. Thanks for any help/advice on this. We worked so hard to get our business on the first page for many keywords and "POOF" we are gone now.
SEO Learn Center | | LinckB0 -
Why did one of our keyword score an F but is first place on Google search? The key word is "Barista Classes Los Angeles"
We're working with an SEO company that provides monthly details on what keywords have improved and needs work. One of our keyowords (Barista Classes Los Angeles) climbed up to number one on Google's search box. However, on SEO Moz, the same keyword got an "F" under the On-Page category. Can someone help us out and define why this is happening? This way, we can adjust and see how we can improve. Thanks!
SEO Learn Center | | ckroaster0 -
I missed the "Making Your Pitch" webinar
When will this be available in the previously recorded section? Thanks Sean
SEO Learn Center | | SeanTheWebGuy0 -
Google's weighting of Page Load speed
Hey, we recently optimised our website page load speed as part of our overall optimisation. Page load speed according to the Google Developers Page Speed test was previously 51 out of 100 and is now 92 out of 100 and this was improved within the last seven days gradually, cached half way through the improvement process. I appreciate this is regarded as only a small part of the whole process, however, I’d be interested to know if anyone has a concrete opinion/proof on whether such a big improvement would actually make a difference to our rankings in the SERPs.
SEO Learn Center | | tdsnet0 -
Machine Learning - Randomness in Search Engine Ranking Algorithms
I believe, sometimes you may 'deserve' the first position, but get #3. And of course, sometimes you may deserve #3 and instead be #1. All due to a 'randomness factor' in search engine algorithms. I've been holding this hypothesis for quite sometime. Really, it started when I learned about SEOmoz using machine learning to better investigate SEO best practices. I suddenly found myself wanting to learn more about machine learning, and the implications of using it for SEO. I'm by no means able to utilize machine learning myself, but I it appears unsupervised learning would have a real chance of cracking search engine algorithms. Hey, even Stuxnet was cracked! Surely Google/ Bing would know (and account for) this, right? We can agree they'd obviously prefer a highly skilled mathematician not be able to crack the code. Therefore, I'm led to believe that search engines use some sort of randomness in their rankings. Maybe not much. Perhaps not all the time. But if a random percentage of search results, had a random variable of sorts included in their calculations... wouldn't that be enough to prevent the vast majority of cracking attempts? Thoughts, opinions, criticism? Thanks.
SEO Learn Center | | DonnieCooper0