I want to parse google and yandex search results for my little website analyzer utility.
so i should send hundreds requests per minute. What is good practice for this issue?
Is google search api a good way?
The Google Search API may not be used for bots. Google will block your utility if you request too much searches.
http://code.google.com/apis/ajaxsearch/terms.html
You agree that when using the Service, You will not, and will not permit users or other third parties to:
Use any robot, spider, site search/retrieval application, or other device to retrieve or index any portion of Google Search Results or to collect information about users for any unauthorized purpose;
Related
I have seen many examples where a search task was launched from application but no examples where application could get some info back from that task.
For example, is it possible to get a list of found sites or URL that user have chosen in task (instead of navigating user to that URL)?
There is no built-in API in WP8 that would allow getting the results of a web search into your app. You should use a 3rd party framework for that.
Bing has an official Web API you can use for that but it starts costing money after 10,000 API calls per month. You can read more about it # https://datamarket.azure.com/dataset/5BA839F1-12CE-4CCE-BF57-A49D98D29A44
Note that Bing did use to have a proper C# API with API keys and what not, but V2.0 of that API has been depercated in favour of the new Azure Data Marketplace service.
Google doesn't have an API for search results. But for the past few years developers have used Google's AJAX webservice to get those. See a C# code sample # http://answers.oreilly.com/topic/2165-how-to-search-google-and-bing-in-c/
I'm new to the mobile development world and right now I'm building an app that uses jQuery mobile and PhoneGap. Here's my logic:
I have a table that contains the users and their addresses. I grab the user address and pass it trough the maps API to catch the location. But I'm doing this for every single record and sometimes the API breaks because the number of simultaneous requests.
How can I know what users are closest to me without running it through the maps API?
Thanks a lot!
Rafael, Google limits the number of requests you can do because it's against their TOS to do batch geocoding without paying for a (rather pricey) business license. You'll need to find a service that allows you to geocode addresses en masse.
I would recommend one like LiveAddress API which will not only geocode the addresses but also confirm the validity and fill out missing or incorrect information. Each request to the API may include up to 100 addresses. There's a non-API version if you just want to verify an existing list of addresses in a spreadsheet or CSV file.
I do work at SmartyStreets and would be happy to help you with any other questions about your addresses.
We're currently integrating Google Drive/Docs access in our mobile Apps and use the Google Document List API for this purpose. Are there any restrictions on the number of requests allowed for single API key?
I can't find any information in the Google API Console as the Document List API is not listed there. I can only activate the Google Drive API (which does not yet support functionalities we need).
The Documents List API does not use API keys in the same way as the newer APIs such as Drive. We (Google) do not give exact quota details for this API, but in general the value is extremely high. You may encounter 503 responses which indicate that you should perform exponential backoff. If, despite this, you are hitting an absolute ceiling, you should contact us and we will investigate, and look to increase your quota.
My question is not about FourSquare API and its functions, but about more simpler details that are not well explained on Foursquare API explanations. Thank you very much in advance if someone wants to help me with this doubts:
Foursquare API is a framework you can use to build applications for mobile devices, above of IOS and Android, so i can imagine that they have API for IOS (Objective-C) and Android (Java), right?
From API Doc: "Be sure to note that although API requests are against api.foursquare.com, OAuth token and authorization requests are against foursquare.com.". Does it mean that if i want to use FourSquare app, the users has to have an account on FourSquare?
From API Doc: "For example, if you write an iPhone application, every user who logs in with their foursquare account will be able to make up to 500 users/* requests and up to 500 venues/* requests, etc." I dont understand this sentence. Does it mean that for example, if you use an API method request like "checkins.add()", this method create two methods? one against api.foursquare.com to monitor the API limit requests, and another to your Web Application Server?
So as a question related to the third one, where do you have to store your database? is it stored on Foursquare cloud database because you are loggin there, or you have to create your own Web Service application with its own SQL database?
From API Doc: "All requests are simple GET or POST requests that return JSON or JSONP respones", so i can imagine that the Web Application Service should understand JSON. Well, my main question is, can i use Ruby on Rails to build the Web Application Service and Web Page frontend? I am seeing that there are some wrapps for RoR designed from third companies, but are not official and doesnt cover all the 2.0 API, just the ones they needed for their services.
If i want to create an app using FourSquare API, what do you advice me to use as a programming language/framework for the Web Service Application? the WSA that has to process the JSON requests and later store them on the database, interaction with users on the WebPage, etc.
i am so sorry if my questions are so simple, but i dont have any other place of this level of expertise.
thank you very very much in advance.
The API is REST/JSON based, which means that any language that can do an HTTP request and parse a string can be used. There are Java and iOS libraries available. But you could use just about anything - curl with bash would be a bit extreme but if that floats your boat...
For some of the APIs (search a venue, for example) you do not necessarily need a FourSquar OAuth user token. For others (like checkin) a FourSquare token is required. For any API calls that require a userid, your users will have to be FourSquare users and "trust" your application with their FourSquare data.
Only requests to FourSquare is counted. So if you do a single call to checkins.add() it counts as one call for the user that is doing the checkin. I wouldn't worry about the limits. As long as you're usage of the API is sensible they will not be a problem. And if they do become a problem and you're doing something extraordinarily cool, the folks at FourSquare might be sympathetic.
You have to create your own web server with your own database to store some information. The OAuth token is one. You probably want to cache venue information here for short periods as well.
Yes, your webapp will need to be able to understand JSON. Ruby has excellent JSON support - look for the json gem.
It is really difficult to suggest a language or framework without knowing what it is that you're trying to do. I wouldn't choose a framework based on the fact that you want to use FourSquare (anything will do) but rather on your experience and the unique features of your application. You mentioned RoR before - that would definitely work.
How can I use read only access to twitter API using OAuth authentication to crawl the twitter data like user timeline, followers, following-list etc. I used to use BasicAuth but now its not supported.
I don't know what platform you're working with, but the "how to connect with Oauth" problem has been solved multiple times on just about every platform. Rather than writing this code yourself (and giving yourself a thousand headaches), you may be better served finding one of the many Twitter libraries out there that already support OAuth.