I am building a simple application where I am searching for tweets related to keywords through twitter apis. Now the API restricts the result to default 15 and maximum 100. I want to know atleast how many total tweets are there for that keyword. I dont want to continue fetching till the results end to get the total count. Is there a simple way without going through the above exercise ?
Thanks in advance.
Related
I want to get all tweets from my country.I tried to get all tweet by using Standart api search method. But it has some rule and limit. I want to do it limitless. Which api method can i use?
If you'd like to get all tweets for the past 7 days you could easily do it with standard API with a geocode cursor search for your country (if you specify language would be better). It will take some time as it will wait 15 minutes everytime it reaches the request limit for 15 min window. You could easily get a million tweets if you run it for 3 - 4 days.
If you need tweets more than 7 days ago you will need premium api but I wouldn't recommend it.
First of all, it is extremely expensive and second, it has a monthly limit of requests. (i am not sure how many but I reckon something about 500). Which means it will actually take you longer than waiting for the 15min rate limit every time.
Use the Premium search API :
https://developer.twitter.com/en/docs/tweets/search/api-reference/premium-search.html
You will be able to search for all tweets since 2006... it may be a lot.
The "query" parameters are listed here : https://developer.twitter.com/en/docs/tweets/search/overview/premium#AvailableOperators so you can filter by lang or with geospatial operators.
TunnelBear offers 1 GB free every month when you tweet about them. I am trying to implement something like this in my web app.
This is how I intend to go about it.
My system will let anyone tweet about my business with some predefined message like "Use Awesome_business to increase profit. Awesome_business is the best!"
A cron job will regularly (every 5 min) fetch the last 10 tweets of all connected twitter accounts
Analyse them to see if they contains keywords like "use", "Awesome_business", "best", "increase", "profit"
Offer discount for every account which tweets will contains keywords listed above.
Does anyone have a better idea about how to go about this?
You could try leveraging Twitter's Streaming API: https://dev.twitter.com/docs/streaming-apis
So you can open a connection and you'll receive tweets as they occur, instead of having to periodically check (plus if there are more than 10 tweets in the last 5 min then you just lost those extras).
Also twitter's search API doesn't allow for specific times, you can only give it a day, so the best you could do is pull all tweets from the last day, and then manually parse the data to find tweets in the specific amount of time from your last poll.
Twitter's API also let's you do hashtag filtering, so you can just grab all tweets with a hashtag relevant to your business, and then reward those people specifically? would make it easier on you so you don't have to write any parsing logic on your end.
I am trying to search keywords in twitter through tweepy.
However, I found it seems like that I can not search the tweets one week before, the code below is the main search code.
for searched_tweets in tweepy.Cursor( API.search,
q = "python",
since = 2014-02-03,
until = 2014-02-04,
lang='en' ).items()
I am not sure whether there is any limited or any better way to search by time, thanks for your help!!
:)
Unfortunately, you can't get tweets older than a week with the twitter search API.
Also note that the search results at twitter.com may return historical results while the Search API usually only serves tweets from the past week. - Twitter documentation.
You can get specific tweets older than a week by looking at individual users or using specific post ids (if you have them) but it's not reasonable to index every single tweet ever to be searchable using the API.
If you need a large time range, you can collect them yourself using the streaming API or check out a service that does (see this dev twitter thread for examples).
I am working on a project which requires getting tweet and user information from twitter. I can't even test the current system because I keep hitting twitter rate limit. Is there any way around it?
Basic information that I am looking to extract from each status is:
Status text
User follower count
User following count
Retweet count
Geo location co-ordinates
I am using Twitter4J API to do this.
Any help will be appreciated. Thanks in advance.
EDIT
I am using twitter's search API to get list of tweets.
One option is to use a Twitter Data Reseller (e.g. GNIP) who can sell tweets.
Another option is to maximize your use of the API. Here are some tips:
Check Rate Limit Status for each API you're use to make sure you don't go over and when the rate limit resets (currently every 15 minutes).
Look at the parameters to make sure you request a count of the maximum number of tweets for that API. e.g. a count can default to 20, but you can set it to 200, depending availability and limits on the specific endpoint. This potentially reduces the number of queries you have to make.
Page your results according to the Twitter's Working with Timelines guidance. Use SinceID and MaxID to make sure you're only requesting new tweets. This could reduce requests by reducing the number of tweets you need (through increasing the opportunity to stay within max count) and reducing the number of requests by avoiding queries for tweets you already have.
Essentially, you want to examine endpoint parameters with a perspective for how to decrease bandwidth and reduce the number of queries you have to make.
My test tweets are not showing up in the twitter search. I have asked several other people to test this as well and only a few show up.
I am trying to track a campaign on the number of people who tweet a certain hashtag. How am I supposed to track it accurately if the search does not give exact results?
Has anyone else experienced this problem?
I read where the API isn't extremely accurate. It only returns results after a crawl, and may not have crawled the entire database when you're searching. The API also doesn't hold on to tweets longer than a few days old.