There is a large number of sites like Twitaholic or Twittergrader that offer rankings of Twitter users depending on the number of followers, influence, etc. I haven't found much information, though, on how do they compute these rankings.
My guess is that they begin with a handful users and keep exploring the followers' graph, while periodically updating the information of the users they already know of.
So the question is: is this the right approach or is there a more trivial way of doing it?
The sites you mention started years ago, and at that time they were given whitelisting by Twitter, which means that they can make tens of thousands of API requests per hour. Twitter no longer gives out new whitelisted accounts, so this type of analysis cannot be done by new sites. New accounts are only allowed to make 350 API requests per hour.
It is in fact possible just to use the Twitter API to examine and remember everything about every user, which is what quite a few sites do. twitter streaming api
Related
I just learn about Twitter for couple of days. I 'm facing a problems in building an app that add a large number of users to block list via Twitter Api (POST blocks/list ). But this Api is only work with one user per request, using it will lead to 'API RATE LIMIT'. That I want here is a method that can add a lot of users to the block list via one request Can anyone help me overcome this problem ? Thank and best regards. Like this site http://blocktogether.org/
You're getting two things confused.
There is GET blocks/list which allows you to receive a list of a blocked users.
On the other hand POST blocks/create allows you to block users from your twitter account. I believe that's the one you're wanting.
Either way they're both rate limited and you'll need to send multiple requests to the twitter API in order to reach whatever number you're looking into adding. As of now, until twitter decides to change it, it's only 1 user at a time.
I am trying to extract data from Twitter. The data includes the tweets and people who retweeted a particular tweet. I have 46,000 tweets and I need to find retweeters for each of the tweet. Further, using Twitter call: retweet/id, you can pass only one id at a time, limiting 15 requests per 15 minutes.
Is there any way to surpass this limit and make unlimited calls?
Not through the REST API, no.
You may want to investigate Twitter's Streaming API to see if the functionality it provides meets your needs. Accessing it is a little more complex than the REST API, but it may be able to help you meet your needs.
You will find people who will tell you to do things like set up dummy accounts and dummy applications. Don't do this. Twitter actively monitors the API for use patterns like this and you will find your applications and IP addresses blacklisted.
I am building a Twitter app and I'll be pulling a big amount of data from the user's timeline. For speed, I need to query the timeline in parallel. My aim is to pull 1000 of user's tweets from the API, but the upper limit of number of tweets per request is set to 200 by the Twitter API. Pagination works by specifying the last (oldest) tweet's ID from the previous request, so I need to know the result of the previous API call to make the next call. This method is not parallelizable. Is there any alternative method for getting the user timeline from the Twitter API where I can make parallel requests (there is the page property, but is deprecated and will be nonfunctional in the near future).
What you have to remember, is that Twitter have a difficult relationship with external developers. Using their API for anything interesting like this is simply not allowed by them.
What you need is access to the Firehose.
However, even if you're willing to pay a million dollars a year - Twitter aren't interested.
You could try getting it from a third party like Gnip but - again - likely to be expensive.
So, essentially, you can't. Twitter just aren't interested in amateur developers doing anything innovative with their platform. Sorry.
We have some quick question about putting together a web crawler to collect some data from Twitter.
For example, if we want to use a few hundreds of user IDs as keywords to collect some necessary Tweets. However, it seems we can only use a limited number of keywords (12?) and we can launch one crawler at one time.
Any tips?
Due to Twitter API restrictions (rate limiting) it is not imaginable to crawl the whole website, unless you have a lot of time or unless you pay in order to have a special access to the twitterhose, which should be the only way to solve the problem you mentioned
I am writing a small app, building stats for twitter users (no of tweets, friends etc). I am using this api
http://api.twitter.com/1/users/show.json?user_id=12345
I can only make 150 calls per hour, which is very very small, given the size of twitter. How do companies that rely on Twitter's API manage to overcome this rate limit?
The 150 API calls is per user per application. Larger companies likely broker deals with Twitter.
You need to get whitelisted to get a far higher rate limit. They are open to all sorts of developers, as long as you give a good reason for what you are developing:
http://twitter.com/help/request_whitelisting
You will easily get whitelisted, just apply. They will accept more or less any reasonable application, but just don't want to leave it 'wide open'. If they dont accept you, and you still want to get your hands on the data, just scrape it.