The Twitter Search API documentation for the attribute result_type specifies three different options: mixed, recent, popular.
The documentation is vague about the differences between recent and popular. Does anyone know how the API uses these attributes? For example, does recent only query the last X hours? Does popular have a threshold for retweet count?
Documentantion: https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets
Popular only returns tweets if they meet a certain threshold, it might be based on favorites or another special algorithm. If a query does not yield any tweet that's not classified as "popular", it won't return any tweets.
I think recent is the default result_type if none is specified.
Related
I'd like to extract all tweets in the Arabic language in all countries.
I modified the code in this tutorial.
This is my search query.
api.search(q="*", count=tweetsPerQry, lang ['ar'],tweet_mode='extended'). I expect to find a very large number of tweets, but I only collected about 7000 tweets.
I checked the content of some of them and I noticed that they are posted in my country even I did not specify the location/Country (Can anyone explain why this happen??).
I tried to know the reason for finding a limited number of tweets, so I modified the query by replacing the lang parameter by geocode to find tweets in a city. I fetched more than 65,000 Arabic tweets. After that, I used the lang parameter with the geocode and I found a very limited number of tweets.
Can anyone help me to know why I'm not able to get a large number of tweets when I used lang parameter?
The free twitter API's are good for small projects, but keep in mind that they don't display all of the tweets. Twitter has paid API's that are much more powerful, though what you are trying to achieve should be possible. I ran the query attached bellow, it seemed to work I was able to find a considerable amount of tweets. This method also seemed to work for #ebt_dev too I think it was just the structure of your request was set out like the stream listener version not the cursor search.
# Search Query change the X of .items(X) to the amount of tweets you are looking for
for tweet in tweepy.Cursor(api.search, q='*',tweet_mode='extended', lang='de').items(9999999):
# Defining Tweets Creators Name
tweettext = str( tweet.full_text.lower().encode('ascii',errors='ignore')) #encoding to get rid of characters that may not be able to be displayed
# Defining Tweets Id
tweetid = tweet.id
# printing the text of the tweet
print('\ntweet text: '+str(tweettext))
# printing the id of the tweet
print('tweet id: '+str(tweetid))
I am trying to connect to streaming API of twitter and retrieve tweets keywords using specific keywords. I am using the phirehose library for the same. It says in the twitter documentation that "commas as logical ORs, while spaces are equivalent to logical ANDs (e.g. ‘the twitter’ is the AND twitter, and ‘the,twitter’ is the OR twitter)."
But I want to search for keywords with AND operator even if there are other words in between. Meaning if we want to search for tweets having Keyword1 AND Keyword2, tweets which have only one keyword should not be retrieved.
Using the settrack function of the phirehose library -
setTrack(array('the , twitter'));
retrieves tweets with either the OR twitter while
setTrack(array('the twitter'));
retrieves tweets with the phrase the twitter and does not retrieve tweets like the busy twitter for example.
Please help.
140dev by Adam Green gives a solution for this by using ``typeenum('words','phrase') NOT NULL DEFAULT 'words'
Please see - http://140dev.com/twitter-api-programming-blog/streaming-api-enhancements-part-2-keyword-collection-database-changes/ and
http://140dev.com/twitter-api-programming-blog/streaming-api-enhancements-part-3-collecting-tweets-based-on-table-of-keywords/
In Tweepy api for twitter I know we can search tweets by -
api.search(q="a and b")
Here it will search for both a and b appearing anywhere in the status in any order. However I need to do same for the Tweepy stream api, Is there any way to do that?
I know there is track field -
stream.filter(track=['a','b'])
But this would return status containing either of a or b, I need both of the keywords in any order.
We could also do like, search for only a then manually filtering out status which contain b, but then we would be discarding huge no. of tweets as stream api gives only 1% of the tweets
Yes, this can be done easily. Looking at the docs for the Twitter API track paramenter:
A comma-separated list of phrases which will be used to determine what Tweets will be delivered on the stream. A phrase may be one or more terms separated by spaces, and a phrase will match if all of the terms in the phrase are present in the Tweet, regardless of order and ignoring case. By this model, you can think of commas as logical ORs, while spaces are equivalent to logical ANDs. For example, ‘the twitter’ is (the AND twitter), and ‘the,twitter’ is (the OR twitter).
By this logic, to filter by a and b:
stream.filter(track=['a b'])
I would like to search for a tweets within a range of between dates by using Twitter API v1.1
let Query query=new QUery(String query);
what is the query thats suits for my question ??
Thanks in advance for reply back.
I suggest setting until to limit your query to an upper date bound, e.g.:
query.setUntil("2014-07-01");
Then step back through the result set, by making subsequent search calls, until you hit your lower date bound.
Be aware that the search API may not contain all Tweets and it may not 'go back' as far as you need. For more information on it, and other query parameters, take a look at Twitter's documentation on searching.
I'd like to retrieve the tweets for given a hashtag and sort them from the most retweeted to the less retweeted.
The closest thing I've found is using the search call and use the type tag:
E.g.: http://search.twitter.com/search.json?q=TheHashTagHere&result_type=popular
However, I'm not sure on how "popular" option works.
For instance, if it finds 100 tweets with that hashtag I believe it should show the X most retweeted tweets, and if none of those tweets have been retweeted then it should show X of them randomly (or sorted in some other way like the most recent).
Unfortunately, if follows some kind of unknown rule to identify what's popular and what not and even hashtags with thousands of tweets might return only one or two results.
I hope I made myself clear. Thanks in advance :)
PS: I'll use PHP but I think that shouldn't affect the question?
Results will sometimes contain a
result_type field into the metadata
with a value of either "recent" or
"popular". Popular results are derived
by an algorithm that Twitter computes,
and up to 3 will appear in the default
mixed mode that the Search API
operates under. Popular results
include another node in the metadata
called recent_retweets. This field
indicates how many retweets the Tweet
has had.
Source (Emphasis are mine)
Just call with result_type=popular and check the recent_retweets node to see how popular it is. result_type=popular will become the default in an upcome release so beware if you omit this parameter.
Results with popular tweets aren't ordered chronologically. *
If you would like to always have results to show, use result_type=mixed: they will have the result_type in the "metadata" section with a value of "recent", and popular results will have "popular". A small reference about result_types:
mixed: Include both popular and real time results in the response.
recent: return only the most recent results in the response
popular: return only the most popular results in the response.
If a search query has any popular results, those will be returned at the top, even if they are older than the other results. *
*[Twitter API Announcements]
This isn't a programmatic method but rather works in the browser with a chrome extension (HackyBird) :
Install the extension
Search for a phrase e.g. #Social (twitter.com/search?q=%23Social)
Click the extension to sort it (you can adjust the ratio of retweets/likes used for sorting in extension options).
P.S. It'll also sort your or any other user's timeline.