Why does Linkscap API request hang while extracting data ?
-
Hi,
I am using LinkScape API to get follow and nofollow links .
I use cron to get data for each url of sitemap.xml.
However while cron is running, the extraction of data hangs on some pages which i later need to delete manually for re starting the execution.
Do anyone have any idea why this is happening ? How can i ignore such pages ?
-
Hi...
Thank you so much for your reply..
I was implementing the changeswhich may work.. but they didn't.
I recently came to know about updates to seomoz api which are like,
- SEOmoz Free API: 1 request for every 10 seconds
- Free Extended and PRO Members: 1 request every 5 seconds
- SEOmoz Site Intelligence API: 1 request every 2 seconds
I think this will help me a lot .. thank you ..
-
Hey Ravi,
Sorry for the delayed response - I wanted to follow up with the engineers to see if they had any suggestions for you.
They agreed the Limit parameter set to 1,000 might be too large to process. Have you tried adjusting that to 300 or even 500? Do you see better success at a lower limit?
Our system will timeout at about 60 seconds so I'm not sure if the hanging is on our end. If dropping the limit size doesn't help, you might want to think about ending the request after about a minute. Sometimes requests that are too long wil timeout, but work fine on a retry as some data will be cached from the previous request.
I hope this information is helpful, but let me know if you're still experiencing issues!
Thanks,
Carin -
Some pages are the pages which are moved or not found. I am trying to ignore such pages by getting http response code. But this process takes large amount of time. Almost 10 minutes for a single url of sitemap.xml.
The url of call is like : (php)
http://lsapi.seomoz.com/linkscape/links/URL HERE?SourceCols=4&TargetCols=4&Scope=page_to_page&Sort=page_authority&Limit=1000&Filter=follow&AccessID=" . $accessID . "&Expires=" . $expires . "&Signature=" . $urlSafeSignature;
Thanks .
-
Hey! This is an issue I haven't heard of before - would you be able to provide anymore information like an example query to the API and some of the pages you are seeing hang?
Thanks!
Carin
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to sift "site search" data from Google Analytics for trends
I apologize in advance if this has been asked a million times but I'm just not able to find anything on it for some reason. Probably the words "site" and "search" come up a lot in this area... Anyhow, my question: How do I find trends in "site search" data from Google Analytics? I set up "site search" a long time ago. I have thousands and thousands of searches people have made on my site logged and squirreled away. The plan was to review them on a weekly basis, find the trends and start writing content to address interests people seem to be having but not finding on our site. Sounded great at the time. The problem I have, of course, is that among my 10,000 searches (many shown in Google Analytics as "no-results:cats and dogs", etc), there are slight differences that make it difficult to total up search trends. Let's say the list is like this: Term | Search Count Cats | 500
Moz Pro | | rtkl
Dogs | 500
Cat | 250
Dog | 250
Cat food | 5
Dog food | 5
Birds | 1
Bird | 1
Cats are great | 1
Cats are really great | 1
Dogs are great | 1
I like birds | 1
Seriously, I like Cats | 1
Turtles | 1 ... 10,000 more entries, every single one only 1 search per term. OK, so it looks like people like Cats and Dogs a lot, but also Birds and Turtles. But maybe there are snake searches. Maybe there are "cat pajamas" searches and variations on all of the above. Who knows what else is really trending in there??? The review of this data is MIND-NUMBING. Especially when you get into plurality and misspellings, this rabbit hole has no bottom. Is there a tool people in the SEO jam use to take a big ole CSV dump and have it magically sorted by at least potential trends? I mean, there's gotta be, right? And I'm silly for not already knowing what it is.0 -
No data in Moz
Hello, Since last tuesday there has been no data in Moz anymore. The error code for the homepage says: '608 : Page not decodable as specified Content-Encoding:' I checked this list http://moz.com/help/pro/why-can-t-rogerbot-crawl-my-site but it doesn't tell me more.
Moz Pro | | MarcelMoz
Hopefully anyone can help me. Thanks, Marcel0 -
Moz vs google data conflict?
Hi there, I am doing an SEO site audit for a client(giveaway, and here is the problem: when performing site:domain.com on google --> 13,800 pages were found When I see this number it seems to be a bit too much compare to the links i checked on integrity(link check for broken links) which gave me a result of 1291. I digged in more into the Google results and saw hundreds(maybe thousands) of pages that are blocked by robots.txt. So I am thinking, ok this is it, thousands of pages can't be crawled by the search engines. Here is the big BUT though, then I check at my moz crawl (see attachment) and no pages are blocked by the SEs, and then look at the dups, only 23 recorded?? Is Moz not crawling properly the 13,800 results that google finds or is this some magical phenomenon happening here? I am really confused here that is why I need some help here! Thank you guys! A990Hu4.png k842AOn.png
Moz Pro | | Ideas-Money-Art0 -
Loss of Google AdWords API
Since August, we've had intermittent access to the Google Adwords API. Like others in this space, they've let us know that our access has been revoked. We've been working to restore this access, but unfortunately, we have not been able to do so. In the meantime we are working hard to find a suitable replacement that is both stable and a good fit for our customers’ needs. We wanted to let our PRO members know, first to apologize, and second to explain how this has impacted your PRO access. We understand how important it is to have all your keyword data in one place, and we're sorry for any disruptions you've experienced from not having this data source in our app.We do want to clarify that this change does not affect your data from Google Analytics. The products affected by this change are our **Keyword Analysis tool and the keyword details and **keyword analysis pages within our web app. We removed this keyword query volume data within PRO a little earlier this year. You can still generate reports, but the metrics previously pulled from the Google Adwords data source is no longer there.The good news is, we don't actually use the Google AdWords data in our calculation of keyword difficulty, so these reports are still 100% accurate—they're just missing the information we usually pull from the API, which you can find on the Google AdWords Keyword Tool here. We're so sorry about this, and understand how frustrating the loss of this data is. We're hoping to have an alternative data source soon, and we will keep you posted as we make progress. Please feel free to comment, ask questions, and get help by replying to this post. Thank you!
Moz Pro | | AaronWheeler7 -
Is the Linkscape Index getting updated?
I know it only gets a big refresh once a month...The last being in the early part of January. But I released a map that went viral (front page of CNET, Wired, Scientific American) that has gotten me hundreds of links from different domains: https://www.google.com/search?q=perceptionbuilder.com+map I got these links in early to mid December and OSE is showing no data at all. Similarly, I have been doing some linkbuilding for clients and links that I got in October and November and those links aren't showing up either. Anyone else experiencing this? Many thanks, Matt
Moz Pro | | coppersix0 -
Changing the Timeframe of Historical Crawl Data
Hello, Just read a great post about the implications of duplicate content for sites after the most recent Panda update: http://www.seomoz.org/blog/duplicate-content-in-a-post-panda-world?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+seomoz+(SEOmoz+Daily+Blog) In the post is an image or crawl data history that shows months, not days or weeks, worth of trending data as it relates to duplicate content. So my question is this: How do I change my view/date range on my own campaigns so that I can view the trailing months of data rather than only what seems to be the past 4 weeks or so? This would really help me identify the impact of some on page changes we've recently made for a client. Many Thanks, Jared
Moz Pro | | surjm0 -
GWT data v SEO Moz data
I'm sure there's a simple explanation but I can't find it- Looking through the GWT avg position data v SEOMoz ranking data and Market Samurai data there is a large discrepancy. For one term that we hold #1, and have held for a month or two GWTis showing that the avg position is 180 (and down 30). Any reason you know of why the GWT avg position data would be so out of whack with our SERP's and other data? Thanks.
Moz Pro | | MrPaulB0 -
When do I see the new Linkscape?
Hi I have seen a Tweet and item on the latest news saying that the Linkscape has been updated. I'm still seeing a report generated at the start of May though. What do I need to do to get my grubby hands on the latest data? Chris
Moz Pro | | P4D0