When using the search query "from:realdonaldtrump -filter:retweets lang:en", when searching for 100 Tweets, I receive 40 different Tweets, and then the other 60 Tweets are exactly the same. This doesn't happen when I have a query such as "computer science -filter:retweets lang:en". Does this have anything to do with how the Twitter API works, or would it be an issue with code?
Related
How to download recent tweets regardless of keyword? I want any recent tweet from the Twitter API version 2. Is it possible? If it is, how to write a query?
For example, it will download tweets containing word cat:
https://api.twitter.com/2/tweets/search/recent?query=cat
but what query to use to use instead of ?? to get tweets for any keyword:
https://api.twitter.com/2/tweets/search/recent?query=??
In API1.1, it was possible to use * as a query, but it seems it is not working for API2. If I commit a query I get the following error: The query query parameter can not be empty.
I think this would return far too many results as it would be all Tweets from all users, all over the world! The "recent" endpoint will limit results to those from the last 7 days but it would still be millions.
You can query just the tweets from a particular account (e.g. Stackoverflow) with:
https://api.twitter.com/2/tweets/search/recent?query=from:stackoverflow
I have a general question regarding twitter APIs in python - is there a way to get the total number of times a particular word, or phrase were tweeted?
Thanks in advance.
You can't get that for the life of Twitter. However, you might be able to the search API to get an idea of how many times over the last 2 weeks, which is the approximate max amount of time the search API goes into the past:
auth = tweepy.auth.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth)
search_results = api.search(q="<your word>")
Then count the number of tweets you get back for an approximation.
For more info, look at the Tweepy Search API. Also, look at Tweepy Cursors for getting more than the default count of tweets.
I'd like to extract all tweets in the Arabic language in all countries.
I modified the code in this tutorial.
This is my search query.
api.search(q="*", count=tweetsPerQry, lang ['ar'],tweet_mode='extended'). I expect to find a very large number of tweets, but I only collected about 7000 tweets.
I checked the content of some of them and I noticed that they are posted in my country even I did not specify the location/Country (Can anyone explain why this happen??).
I tried to know the reason for finding a limited number of tweets, so I modified the query by replacing the lang parameter by geocode to find tweets in a city. I fetched more than 65,000 Arabic tweets. After that, I used the lang parameter with the geocode and I found a very limited number of tweets.
Can anyone help me to know why I'm not able to get a large number of tweets when I used lang parameter?
The free twitter API's are good for small projects, but keep in mind that they don't display all of the tweets. Twitter has paid API's that are much more powerful, though what you are trying to achieve should be possible. I ran the query attached bellow, it seemed to work I was able to find a considerable amount of tweets. This method also seemed to work for #ebt_dev too I think it was just the structure of your request was set out like the stream listener version not the cursor search.
# Search Query change the X of .items(X) to the amount of tweets you are looking for
for tweet in tweepy.Cursor(api.search, q='*',tweet_mode='extended', lang='de').items(9999999):
# Defining Tweets Creators Name
tweettext = str( tweet.full_text.lower().encode('ascii',errors='ignore')) #encoding to get rid of characters that may not be able to be displayed
# Defining Tweets Id
tweetid = tweet.id
# printing the text of the tweet
print('\ntweet text: '+str(tweettext))
# printing the id of the tweet
print('tweet id: '+str(tweetid))
I am using https://api.twitter.com/1.1/search/tweets.json?q=' . $_POST['keyword'] . ' to get the tweets of searched keyword.
I am able to get till 100 tweets but i need to display full 3,200 tweets.
how can i do with this?
how to get video from the tweets?
For videos, Twitter still working on this and for time being video results are not coming in "Search API". We can track this here(see at few last comments): https://twittercommunity.com/t/twitter-video-support-in-rest-and-streaming-api/31258/38
For more than 100 tweets, you can call API for next tweets as per below:
Find the the highest id(data[data.length - 1].id) in the twitter results that you just retrieved with your query
perform the same query with the "since_id" option set to the id you just found.
Ex:
https://api.twitter.com/1.1/search/tweets.json?q=nature&since_id=602821207268921300
I'm working on a research project which analyses closure patterns in social networks.
Part of my requirement is to collect followers and following IDs of thousands of users under scrutiny.
I have a problem with rate limit exceeding 350 requests/hour.
With just 4-5 requests my limit is exceeding - ie, when the number of followers I collected exceeds the 350 mark.
ie, if I have 7 members each having 50 followers, then when I collect the follower details of just 7 members, my rate exceeds.(7*50 = 350).
I found a related question in stackoverflow here - What is the most effective way to get a list of followers using Twitter4j?
The resolution mentioned there was to use lookupUsers(long[] ids) method which will return a list of User objects... But I find no way in the API to find the screen names of friends/followers of a particular "User" object. Am I missing something here.. Is there a way to collect friends/followers of thousands of users effectively?
(Right now, I'm using standard code - Oauth authentication(to achieve 350 request/hour) followed by a call to twitter.getFollowersIDs)
It's fairly straightforward to do this with a limited number of API calls.
It can be done with two API calls.
Let's say you want to get all my followers
https://api.twitter.com/1/followers/ids.json?screen_name=edent
That will return up to 5,000 user IDs.
You do not need 5,000 calls to look them up!
You simply post those IDs to users/lookup
You will then get back the full profile of all the users following me - including screen name.