Thoughts on scraping SERPs and APIs
-
I'm perplexed about a couple of things.
Google says that scraping keyword rankings is against their policy from what I've read. Bummer. We comprise a lot of reports and manual finding and entry was a pain. Enter Moz! We still manually check and compare, but it's nice having that tool. I'm confused now though about practices and getting SERPs in an automated way. Here are my questions
- Is it against policy to get SERPs from an automated method? If that is the case, isn't Moz breaking this policy with it's awesome keyword tracker?
- If it's not, and we wanted to grab that kind of data, how would we do it? Right now, Moz's API doesn't offer this data. I thought Raven Tools at one point offered this, but they don't now from what I've read. Are there any APIs out there that we can grab this data and do what we want with it? (let's day build our own dashboard)?
Thanks for any clarification and input!
-
You're much too kind Happy New Year!
-
Hi Rand. Anyone ever tell you you're awesome?
I just did. Thanks bud
-
Hi Boogily,
A) In terms of ToS, I'm not 100% clear. I know that Google has been strict about automated rank checking programs installed on a user's local machine, and they've enforced that a bit through technical and guidelines means. However, they've been unclear about companies like Moz, Searchmetrics, GetStat, AuthorityLabs, Conductor, Brightedge, Yield, RankAbove, and hundreds of others which provide this data in their software. I suspect they don't love it, but tolerate it and haven't taken action for over a decade now.
B) On the API front, check with GetStat and AuthorityLabs. I believe they both have rank data offerings from an API standpoint. Moz doesn't offer that and probably won't in the future, either.
Cheers!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Does anyone know why the MOZ SERP analysis has become so slow that it's not even useful?
Does anyone know why the MOZ SERP analysis has become so slow that it's not even useful?
Moz Pro | | PLEXI3 -
How to use the Basic SERP to figure out what keywords to target
I'm looking at the SERP report trying to figure out what keywords I should try to rank for and I am having difficulty understanding if I could be competitive in the space. For example I have a new website that has a PA1 and DA1 rank I plan on developing great content and then working on securing quality backlinks for that content. In order to start this process I'm doing my keyword research and I am finding keywords, both short and longtail that I would like to target via Google Keyword Planner. I'm punching them in to see the competition and SERP. This is where I fall short. How do I use this information to identify or rule out what to pursue. I guess I am looking for some criteria or guide to help me begin to figure out or filter out what is the best keyword to pursue that will give me the best odds of success. If anyone can help me I would be eternally greatfull. We are a small company (startup) and it is up to me to make this work. YES - I have read everything that I can find - guides, and I have not run accross an answer. Thanks in advance
Moz Pro | | DDWeddings1 -
Rank tracking in Excel via MOZ API?
Hi there, i hope someone knows the solution to this problem. I created/ copied the "advanced Excel Keyword Analysis Report" that Dan Peskin was kind enough to share, you can find it here: http://moz.com/blog/how-to-build-an-advanced-keyword-analysis-report-in-excel. Now i m looking for a solution to rank my Keywords by using an API, preferably the MOZ API. My keywords are alligned in a column "Keword List", which will be regularly updated. Next to this column i would like to have a column with the corresponding rank. Is there a automatic solution, or do i have to do it manually?
Moz Pro | | Juri.Rabiner0 -
I cant generate my Mozscape API
I have tried to generate my Mozscape API numerous times after waiting at least 20 mins between. Any help would be great.
Moz Pro | | MLMLEADZONE1 -
Video SERP and Seomoz ranking
It seems that my keywords that lead to videoresults in SERP, is not being ranked on Seomoz! How come when i rank #3 on a certain keyword with a videoresult Seomoz is displaying me as #15 (which is the second of my pages in SERP)? What is the reason for not counting the video-result in the ranking-tool?
Moz Pro | | alsvik0 -
SEOmoz ranking report SERP
I was wondering how SEOmoz the SERPs tracks? e.g. the SERP of keyword in the Google US report, doesn't give me the same result as https://www.google.com/search?pws=0&gl=us&q=keyword Does SEOmoz check the google.com SERPs from several locations and calculate an average?
Moz Pro | | Teklan0 -
Opensiteexplorer details in API
Is there a way to access opensiteexplorer result details via seomoz's API? Most interested in the social media results
Moz Pro | | fuzzlepop0 -
Some thoughts on MozTrust based on OSE Findings X ref'd with SERPS
I've been doing a bit of competitor analysis for a client using OSE. There are a group of about 4 websites (our clients website included) that all dominate the sector with none of the 4 clearly out in front (call this GROUP A). Then there are another group of about 5 websites, which come lower in the SE's consistently than the top 4 (Call this GROUP B) **I've been doing some analysis in OSE: ** ALL GROUP B Websites outrank all of GROUP A websites in the OSE Metrics (Including Trust Rank). I did some analysis on the backlinks in Group A VS Group B Group A - Generally a mixture of ok links from blog posts, sponsorship, and ok directories. Group B - As A, but with fewer numbers of links from quality blogs PLUS A high level of spammy links ( .edu and .gov spam filled pages), very low quality, almost non legible blog posts on MFA sites (think Digital Point sellers). From the above it is clear that the OSE metrics are out of whack with the real SE results. Clearly OSE has a few problems with working out what are spammy links and what are decent. Obviously google also has issues with working this out, so I am not surprised that OSE also does - but that doesn't solve the issue. This is a general discussion - so I would just throw in a few thoughts on how OSE may possibly try are overcome some of these issues : 1/. % Trust Links vs Non trust Links:
Moz Pro | | James77
Add in a metric to Trust Rank where the number of links close to trusted sites are also compared to the number of links not close to trusted sites. If you see a very high ratio of links from sites that are not close to trusted sites, it is a strong indicator of spammy links. 2/. Use seed "Non Trusted" sites to create a negative Trust Rank
Use something like a reverse of the "trusted sites" theory, but taking a load of very clearly spammy / link manipulative sites and work out in terms of links connections how far the site is away from these sites. Thoughts???0