Why does Linkscap API request hang while extracting data ?
-
Hi,
I am using LinkScape API to get follow and nofollow links .
I use cron to get data for each url of sitemap.xml.
However while cron is running, the extraction of data hangs on some pages which i later need to delete manually for re starting the execution.
Do anyone have any idea why this is happening ? How can i ignore such pages ?
-
Hi...
Thank you so much for your reply..
I was implementing the changeswhich may work.. but they didn't.
I recently came to know about updates to seomoz api which are like,
- SEOmoz Free API: 1 request for every 10 seconds
- Free Extended and PRO Members: 1 request every 5 seconds
- SEOmoz Site Intelligence API: 1 request every 2 seconds
I think this will help me a lot .. thank you ..
-
Hey Ravi,
Sorry for the delayed response - I wanted to follow up with the engineers to see if they had any suggestions for you.
They agreed the Limit parameter set to 1,000 might be too large to process. Have you tried adjusting that to 300 or even 500? Do you see better success at a lower limit?
Our system will timeout at about 60 seconds so I'm not sure if the hanging is on our end. If dropping the limit size doesn't help, you might want to think about ending the request after about a minute. Sometimes requests that are too long wil timeout, but work fine on a retry as some data will be cached from the previous request.
I hope this information is helpful, but let me know if you're still experiencing issues!
Thanks,
Carin -
Some pages are the pages which are moved or not found. I am trying to ignore such pages by getting http response code. But this process takes large amount of time. Almost 10 minutes for a single url of sitemap.xml.
The url of call is like : (php)
http://lsapi.seomoz.com/linkscape/links/URL HERE?SourceCols=4&TargetCols=4&Scope=page_to_page&Sort=page_authority&Limit=1000&Filter=follow&AccessID=" . $accessID . "&Expires=" . $expires . "&Signature=" . $urlSafeSignature;
Thanks .
-
Hey! This is an issue I haven't heard of before - would you be able to provide anymore information like an example query to the API and some of the pages you are seeing hang?
Thanks!
Carin
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Receiving incorrect data for eFurnitureHouse.com recently migrated from Network Solutions to Big Commerce
We have recently migrated www.eFurnitureHouse.com from Network Solutions to Big Commerce. The Crawl data we are receiving from MOZ is stating (missing meta description) for product categories. However, the meta description is there. On WMT ( HTML improvements) we are receiving 1000s of (Pages with duplicate meta descriptions). However, when you click on older URL it correctly redirect to new. On WMT ( HTML improvements) we are also receiving 1000s of ( pages with duplicate title tags) We would greatly appreciate any help here. Regards, Tony
Moz Pro | | OCFurniture0 -
Perplexed - Errors increasing, moz rank dropping, conflicting data from other sources. Please Help.
Hi there, I'm in a bit of a bind and could seriously do with some help. For the past month I've been working with a client to resolve errors onsite. In that period my moz rank has plummeted and my errors (dupe title & content) have increased dramatically. The correlation makes sense, the increase - not so much. Here's why, screaming frog is reporting that the majority of duplicate titles have been removed or dealt with. I've also got the assurance of the developer that the suggested changes are being implemented. Unfortunately, I'm not familiar with the CMS - umbraco - and this is the first time I'm working with this particular developer, so I'm not sure how to gauge progress without using moz tools. So, here are my immediate questions: 1. Why would I get different results from Screaming Frog and Rogerbot? 2. Has anyone here been in a similar situation and could they recommend areas to watch for errors creeping in? 3. Is there a way of identifying which errors have occurred when? Narrowing it down to the week each error occurred would be good enough. 4. Is anybody aware of any inherent SEO flaws in umbraco or common SEO mistakes made using the CMS? 5. is there anything I could provide you with to help you help me? 6. Do you have a suggestion? I'm pretty stuck. Thanks in advance.
Moz Pro | | KJDMedia0 -
4XX (Client Error) Report - Referrer URL Feature Request
Why not allow me to click on the individual 404 errors in the 4XX (Client Error) report and allow me to actually see the where the broken links are coming from (as a hyperlinked url) so I can locate/click and fix them? Providing the referring url w/hyperlink will stop me from having to jump between the report and searching in site explorer, or downloading the complete error report and sorting.
Moz Pro | | WaterGuy0 -
Using api to find likes, tweets and shares
Hi all, I have a question regarding using the API to get data on a list of urls and not one by one. I am not a developer so I might not grasp the API so well. Basically I have a list of urls, what I need from each is the following information: <colgroup><col width="98"><col width="89"><col width="92"><col width="111"><col width="99"><col width="50"><col width="64"></colgroup>
Moz Pro | | rightmove
| Page authority | linking root domains | total links | Facebook Shares | Facebook likes | tweets | Google+1's | I found a google doc that gives me SOME of the data but not all (also it might be old as it uses the linkscape call (I dont know if thats changed)) Any idea how I can tune my googledoc spreadsheet to include the additional metrics? Thanks 🙂0 -
How do I retrieve crawl and ranking data about a site from the past?
Hey. One of my main clients has asked to see the crawl data and rankings data for the past eight months. He wants to have tangible evidence of the effects of Penguin. I would like that info too. Is it possible to retrieve that information on a weekly crawl and ranking basis through SEO Moz and if so, how do you do it? I simply want to show a graph, timeline and brief explanation across several main keywords... Help me as you guys always do - You rock Best Ben
Moz Pro | | creativeguy0 -
Can I calculate "Keyword Difficulty" metric using Mozscape API data?
We already have a web application that pulls certain metrics about websites using the Mozscape API, but we are wanting to extend the usefulness of this application to enable users of the app to pull "Keyword Difficulty" metrics in bulk, instead of one at a time (or 5 at a time). I wouldn't mind the 5 at a time limitation if we could just automate the API calls and let the tool pull data for 50 or so keywords without user-interaction. I know that it's a "formula", but I don't know what SEOMoz uses for it's formula. Has anyone figured out a way to calculate this, based on the Mozscape API data? Has anyone ever tried to reverse engineer this metric?
Moz Pro | | brchap0 -
How does SEOMoz get ranking data? is it automated queries to Google?
Google's terms don't allow automated queries without Google's permission. https://support.google.com/webmasters/bin/answer.py?hl=en&answer=66357 How does SEOMoz get ranking data then? Is it sending automated queries with or without Google's permission? is it violating Google's guideline? Will it affect my site performance as far as rankings?
Moz Pro | | huangyouyou1170 -
Why is Open site Explorer showing: No Data Available for this URL
Hi there, Im having a few problems getting my site www.incarmotorfactors.co.uk up and running on SEOMoz and im not sure what im doing wrong.Firstly seomoz shows 2 links for my site... Which is wrong. Google shows alot more. However the most noticable problem so far is Opensite explorer. When i type in the web address it shows "No Data Available for this URL" The site is more then a year old and has a few links, can anybody tell me what the problem may be?
Moz Pro | | Ev840