I'd like to extract all tweets in the Arabic language in all countries.
I modified the code in this tutorial.
This is my search query.
api.search(q="*", count=tweetsPerQry, lang ['ar'],tweet_mode='extended'). I expect to find a very large number of tweets, but I only collected about 7000 tweets.
I checked the content of some of them and I noticed that they are posted in my country even I did not specify the location/Country (Can anyone explain why this happen??).
I tried to know the reason for finding a limited number of tweets, so I modified the query by replacing the lang parameter by geocode to find tweets in a city. I fetched more than 65,000 Arabic tweets. After that, I used the lang parameter with the geocode and I found a very limited number of tweets.
Can anyone help me to know why I'm not able to get a large number of tweets when I used lang parameter?
The free twitter API's are good for small projects, but keep in mind that they don't display all of the tweets. Twitter has paid API's that are much more powerful, though what you are trying to achieve should be possible. I ran the query attached bellow, it seemed to work I was able to find a considerable amount of tweets. This method also seemed to work for #ebt_dev too I think it was just the structure of your request was set out like the stream listener version not the cursor search.
# Search Query change the X of .items(X) to the amount of tweets you are looking for
for tweet in tweepy.Cursor(api.search, q='*',tweet_mode='extended', lang='de').items(9999999):
# Defining Tweets Creators Name
tweettext = str( tweet.full_text.lower().encode('ascii',errors='ignore')) #encoding to get rid of characters that may not be able to be displayed
# Defining Tweets Id
tweetid = tweet.id
# printing the text of the tweet
print('\ntweet text: '+str(tweettext))
# printing the id of the tweet
print('tweet id: '+str(tweetid))
Related
Here is my example query. It specifies that Tweets must be:
Written in English
Tweeted between 23Jan2010 and 24Jan2010
Have at least 100 "favorites" (likes)
My idea is to use something like the binary search algorithm to narrow down the minimum number of likes the Tweet has. Once only one Tweet is returned by a query, I'll know it is the Tweet with the most likes. The problem is, min_faves--the value that specifies the minimum number of likes--doesn't seem to work. Look at this query. It specifies min_faves as 100. As you can see, this Justin Bieber Tweet appears. It has 1.6k likes. Now, when I attempt to increase the min_faves value to 300 (to narrow down the most liked Tweet), the Justin Bieber Tweet is excluded! I don't know if I am not understanding the query system correctly, or if it is not working, but this seems incorrect. The Justin Bieber Tweet should show up, as it has more than 300 likes. This is just one example of how it doesn't seem to work.
Perhaps this is ocurring because, within the specified time range, the Justin Bieber Tweet did not have enough likes to meet the requirements. This would be very good for me, as I am trying to find the most liked Tweet on that particular day, and not the Tweet with the most likes right now that happens to have been posted on that particular day.
But, I do not believe this is the case. For instance, this query includes 3 Tweets from "Rev Run" when min_faves is set to 249, but returns 0 Tweets when min_faves is set to 250. I doubt that these Tweets all had exactly 249 likes on that day (as implied by these symptoms).
Does anyone either:
Understand why these results occur and how I can use this method to find the most liked Tweet of a particular day
Know of a better, alternative way I can find the most liked Tweet of a particular day
Thank you all
#sinanspd requested an example from 2018:
Here is a search with min_faves at 300k. It includes a post with 769k likes and a post with 479k likes. When the query's min_faves is bumped up to 400k neither are returned.
I would like to search for a tweets within a range of between dates by using Twitter API v1.1
let Query query=new QUery(String query);
what is the query thats suits for my question ??
Thanks in advance for reply back.
I suggest setting until to limit your query to an upper date bound, e.g.:
query.setUntil("2014-07-01");
Then step back through the result set, by making subsequent search calls, until you hit your lower date bound.
Be aware that the search API may not contain all Tweets and it may not 'go back' as far as you need. For more information on it, and other query parameters, take a look at Twitter's documentation on searching.
I'm getting different results between the searches I run on on twilio.com and the searches I run through the Ruby gem helper.
Here's a sample search:
Here's a search with the same zip code in a Rails console, returning an empty array:
> #twilio_client.account.available_phone_numbers.get('US').local.list({in_postal_code: "19428"})
=> []
These searches were conducted less than a minute apart.
Is this an issue with the REST API, the Ruby helper gem, or my search query?
Ricky from Twilio here.
The code you're using is correct to retrieve phone numbers restricted to a postal code and I can see why you would expect it to match up with the search on Twilio.com. In our search we're actually doing the lookup a bit differently. You can create an experience similar to ours by using a different set of filters to retrieve phone numbers.
#numbers = #client.available_phone_numbers.get('US').local.list(
near_lat_long: '40.6928,-73.9903',
distance: '5'
)
Since phone numbers within a postal code can be sparse we can ensure results by expanding our search to be near the location a user searches for.
I am trying to create a yahoo pipe that takes ideally takes all tweets tweeted at any point in time and filters down by a number of attributes to then display a filtered feed.
Basically in order this is what I want to happen:
Get a feed of all tweets at any one time.
Filter tweets by geolocation origin, i.e. UK,
Filter by a number of of different combinations of keywords.
Output as an RSS feed (though this isn't really the crucial stage as Yahoo Pipes takes care of this anyway)
Disclaimer: of course I understand that there are limits to the amount of tweets that could come through etc but I would like to cast the input net as wide as possible.
I have managed to get stages 3 & 4 working correctly and for the time being I am not really worrying about step 2 (although if you have any suggestions I am all ears), but stages 1 is where I am struggling. What I have attempted is using a Fetch Feed module with the URL - http://search.twitter.com/search.atom?q=lang:en - however it seems that this only pulls 15 tweets. Is there any way that I can pull more than 15 tweets every time the pipe is run, otherwise I think this may all be in vain.
FYI, here is the link to the pipe as it stands - http://pipes.yahoo.com/ludus247/182ef4a83885698428d57865da5cf85b
Thanks in advance!
I'd like to retrieve the tweets for given a hashtag and sort them from the most retweeted to the less retweeted.
The closest thing I've found is using the search call and use the type tag:
E.g.: http://search.twitter.com/search.json?q=TheHashTagHere&result_type=popular
However, I'm not sure on how "popular" option works.
For instance, if it finds 100 tweets with that hashtag I believe it should show the X most retweeted tweets, and if none of those tweets have been retweeted then it should show X of them randomly (or sorted in some other way like the most recent).
Unfortunately, if follows some kind of unknown rule to identify what's popular and what not and even hashtags with thousands of tweets might return only one or two results.
I hope I made myself clear. Thanks in advance :)
PS: I'll use PHP but I think that shouldn't affect the question?
Results will sometimes contain a
result_type field into the metadata
with a value of either "recent" or
"popular". Popular results are derived
by an algorithm that Twitter computes,
and up to 3 will appear in the default
mixed mode that the Search API
operates under. Popular results
include another node in the metadata
called recent_retweets. This field
indicates how many retweets the Tweet
has had.
Source (Emphasis are mine)
Just call with result_type=popular and check the recent_retweets node to see how popular it is. result_type=popular will become the default in an upcome release so beware if you omit this parameter.
Results with popular tweets aren't ordered chronologically. *
If you would like to always have results to show, use result_type=mixed: they will have the result_type in the "metadata" section with a value of "recent", and popular results will have "popular". A small reference about result_types:
mixed: Include both popular and real time results in the response.
recent: return only the most recent results in the response
popular: return only the most popular results in the response.
If a search query has any popular results, those will be returned at the top, even if they are older than the other results. *
*[Twitter API Announcements]
This isn't a programmatic method but rather works in the browser with a chrome extension (HackyBird) :
Install the extension
Search for a phrase e.g. #Social (twitter.com/search?q=%23Social)
Click the extension to sort it (you can adjust the ratio of retweets/likes used for sorting in extension options).
P.S. It'll also sort your or any other user's timeline.