I have a list of 5500 tweet ids. For each tweet id, I'm downloading the associated tweet text. The ids are non-sequential and from many different users.
The only API option I see for pulling a specific tweet is GET statuses/show/:id.
That gives only a tweet at a time. With rate limiting of 350 API calls / hr, that means ~16 hrs to download the data.
Is there an API call or better technique that I'm missing?
You can buy tweets from GNIP.
No. I'm afraid none. There's no way to bypass twitters 350 API Calls per hour for authenticated users.
As far as I understood, Twitter API has an end point to fetch up to 100 tweets per request. You can find the details at: https://dev.twitter.com/rest/reference/get/statuses/lookup
I use tweepy's statuses_lookup method: http://tweepy.readthedocs.org/en/v3.1.0/api.html#timeline-methods So, you can retrieve tweets much faster than requesting one tweet at a time.
Related
TunnelBear offers 1 GB free every month when you tweet about them. I am trying to implement something like this in my web app.
This is how I intend to go about it.
My system will let anyone tweet about my business with some predefined message like "Use Awesome_business to increase profit. Awesome_business is the best!"
A cron job will regularly (every 5 min) fetch the last 10 tweets of all connected twitter accounts
Analyse them to see if they contains keywords like "use", "Awesome_business", "best", "increase", "profit"
Offer discount for every account which tweets will contains keywords listed above.
Does anyone have a better idea about how to go about this?
You could try leveraging Twitter's Streaming API: https://dev.twitter.com/docs/streaming-apis
So you can open a connection and you'll receive tweets as they occur, instead of having to periodically check (plus if there are more than 10 tweets in the last 5 min then you just lost those extras).
Also twitter's search API doesn't allow for specific times, you can only give it a day, so the best you could do is pull all tweets from the last day, and then manually parse the data to find tweets in the specific amount of time from your last poll.
Twitter's API also let's you do hashtag filtering, so you can just grab all tweets with a hashtag relevant to your business, and then reward those people specifically? would make it easier on you so you don't have to write any parsing logic on your end.
I am working on a project which requires getting tweet and user information from twitter. I can't even test the current system because I keep hitting twitter rate limit. Is there any way around it?
Basic information that I am looking to extract from each status is:
Status text
User follower count
User following count
Retweet count
Geo location co-ordinates
I am using Twitter4J API to do this.
Any help will be appreciated. Thanks in advance.
EDIT
I am using twitter's search API to get list of tweets.
One option is to use a Twitter Data Reseller (e.g. GNIP) who can sell tweets.
Another option is to maximize your use of the API. Here are some tips:
Check Rate Limit Status for each API you're use to make sure you don't go over and when the rate limit resets (currently every 15 minutes).
Look at the parameters to make sure you request a count of the maximum number of tweets for that API. e.g. a count can default to 20, but you can set it to 200, depending availability and limits on the specific endpoint. This potentially reduces the number of queries you have to make.
Page your results according to the Twitter's Working with Timelines guidance. Use SinceID and MaxID to make sure you're only requesting new tweets. This could reduce requests by reducing the number of tweets you need (through increasing the opportunity to stay within max count) and reducing the number of requests by avoiding queries for tweets you already have.
Essentially, you want to examine endpoint parameters with a perspective for how to decrease bandwidth and reduce the number of queries you have to make.
I have IDs from many tweets, and I'd like to fetch their full information from Twitter in order to do some data analysis. The obvious API method (https://dev.twitter.com/docs/api/1/get/statuses/show/:id) appears to take only one ID at a time. This is a problem because the number of tweets we need to analyze is well more than the API limit of 350 calls per hour.
Thus: is there some way to get full information for a set of tweet IDs, not just one, or alternately to submit many REST calls in the same HTTP request and have it count only once against the API limit?
There's unfortunately no bulk lookup offered for Tweets. You'll need to perform requests one at a time and scope your project to cope with the rate limitations. If you have friends who would like to help you, you could potentially ask them to authorize your application and leverage their permission to gain access to more requests.
So, I am currently working on an application that allows me to search journalists and view their contact details including, among other things, their latest tweets.
Calling the Twitter API is no problem at all, and I have a list of tweets for the current journo showing up fine. However, I am trying to decide the best approach to seeing if I (the user logged into our application) have retweeted any of the journo's tweets, so that I can display the tweet as having been retweeted.
Looking at the API documentation, I am looking into the following options:
1 - Use /statuses/retweeted_by_me to get the list of tweets that I
have retweeted, specifying max_id to be the highest id from the
list of tweets for the journo I am looking at. What I am not sure of
here is if the max_id is going to limit by the max id of my tweets or
the max id of the retweeted tweets.
2 - Use /statuses/:id/retweeted_by for each of the journo's tweets to
get a list of all the users who have retweeted that tweet and to check to
see if my user is in the returned list of users. Downfalls of this is that there
could be thousands of retweeters and I can only return a maximum of
100 at a time. That could mean a lot of requests just to find if I
retweeted one tweet. This also means that I need to do this for each
of the journo's tweets as opposed to getting the list of all my
rewteets above.
3 - Use /statuses/retweets/:id to get a list of the retweets for
each of the journo's tweets. This option has the same limitations as
#2.
I'm currently leaning towards #1, however, I'm not sure as yet if this will work. I need to do some more investigation. In the mean time, my question to you is:
What is the best approach to working out if I have retweeted a tweet from another user?
Ok, so I must have been having a brain fart moment when looking into this yesterday. It occurred to me this morning that you can simply make an authenticated call to /statuses/user_timeline which will set the retweeted property to true if you have retweeted that tweet.
Simples.
I know that I can use "users/show" and get "followers_count" or I can do "followers/ids" and count the number of IDs returned, but both of these methods are rate-limited at 150 requests per hour when anonymous and 350 when signed w OAuth.
The program I'm writing uses the Twitter Search API to look for all mentions of a hashtag. I'm using the Search API and not the Streaming API because I need to look for historical tweets, not just real time.
When I find a tweet that contains the hashtag, I want to save the user's handle, tweet ID, time of tweet, and the number of followers that user has. Since the number of followers per user isn't returned with the Search API, I need to use another API call for that. That extra call is what's causing me trouble.
Are there any more efficient ways to get the number of followers for more than 350 users per hour? (There are a TON of tweets coming in...)
Your only option is GET users/lookup which supports fetching up too 100 user objects in a single request. Authentication is required so you will be allowed 35000 user objects/hour. If that still isn't enough should look into queueing the requests.