I have been using tweepy to scrape pieces of information from twitter like tweets with a particular keyword, user id, favorite count, retweeters id, etc. I tried to fetch user id who has retweeted certain tweets. After going through tweepy documentation I found the way to do this is.
API.get_retweets(id, *, count, trim_user)
I have gone through similar questions here ,but I could not figure out how to retrieve retweet ids?
number_of_tweets=10
tweets = []
like = []
time = []
author=[]
retweet_count=[]
tweet_id=[]
retweets_list=[]
retweeters_list=[]
for tweet in tweepy.Cursor(api.search_tweets,q='bullying',lang ='en', tweet_mode="extended").items(number_of_tweets):
tweet_id.append(tweet.id) # Instead of i._json['id']
tweets.append(tweet.full_text)
like.append(tweet.favorite_count)
time.append(tweet.created_at)
author.append(tweet.user.screen_name)
retweet_count.append(tweet.retweet_count)
retweets_list = api.get_retweets(tweet.id)
for retweet in retweets_list:
retweeters_list.append(retweet.user.id)
# print(retweet.user.id)
You should chose more explicit names for your variables (i? j?), that would help you (and the helpers here) to check the logic of your code.
Besides, what you want to achieve and what does not work is pretty unclear.
Does it work if you remove the first inner loop? (I don't understand its purpose)
for tweet in tweepy.Cursor(api.search_tweets,q='bullying',lang ='en', tweet_mode="extended").items(number_of_tweets):
tweet_id.append(tweet.id) # Instead of i._json['id']
tweets.append(tweet.full_text)
like.append(tweet.favorite_count)
time.append(tweet.created_at)
author.append(tweet.user.screen_name)
retweet_count.append(i.retweet_count)
# No need for the first inner loop
retweets_list = api.get_retweets(tweet.id)
for retweet in retweets_list:
print(retweet.id) # If you want the retweet id
print(retweet.user.id)
If not, please give more debug information (what is the problem? What do you want to do? Is there any error message? Where? What have you tried? Show some printed variables etc.).
Related
I'm tracking all mentions of #UN with Tweepy using Twitter stream v1 API. However, I'm also getting all mentions of usernames containing #UN such as #UN_Women. I could filter them out in a post-processing step but this seems very inefficient.
Is there any way to avoid this?
This is my code:
class MyStreamListener(tweepy.StreamListener):
def on_status(self, status):
print(status.text)
myStreamListener = MyStreamListener()
myStream = tweepy.Stream(auth = api.auth, listener=myStreamListener())
myStream.filter(track=['#UN'])
Using follow instead of track should work. With follow, you supply a list of user IDs:
myStream.filter(follow=['14159148'])
I don't know if tweepy provides any further functionalities to avoid this. But what you can do here is, filter out the results while saving it to your database or csv.
Check the json response and look for entities object and in that check for user_mentions and screen_name. Save only the one with your desired screen_name
I am using TwitterKit in orther to show my followers list. So I am using this GET request:
https://dev.twitter.com/rest/reference/get/followers/list
The problem is it only returns a list of 20 followers. Does anybody of you know how to manage this problem ? I need it to return all of them.
Any idea ?
Thank you very much
From reading the documentation, this call returns a lost of followers. The default number of the list of followers it return is 20 thus you're getting the 20 contacts only. This API have a parameter called count where you can specify the number of followers you want to retrieve, the maximum is 200.
However there is a call that can reurn all the IDs for the followers: https://dev.twitter.com/rest/reference/get/followers/ids
This will return all followers IDs up to 5000 ID, however it will send ID only and not all the information of the followers. here is the documentation for this API: https://dev.twitter.com/rest/reference/get/followers/ids
Hope this helps!
I'm confused about the friends method in the Twitter gem. I see that Twitter::REST::FriendsAndFollowers#friends uses a GET friends/list request and the cursor_from_response_with_user method.
The documentation for GET friends/list says that requests are limited to 20 users. I assume this means that 20 friends will be returned per request. But, say I am following 22 people and I use the following:
twitter_client = Twitter::REST::Client.new { [my credentials here] }
twitter_client.friends
This returns an array of all 22 friends. I didn't do anything to mess with cursors, so why am I getting all 22? On to my main question...
In my app, when someone imports their friends, I'm iterating over them and creating some other records. Something like this
twitter_client.friends.each do |friend|
SomeModel.do_what_you_need_to_with(friend)
AnotherModel.do_something_else_with(friend)
end
Let's say someone has 5001 Twitter friends. (Impressive!) I'm thinking this is going to be a problem with rate limiting, etc. What's the safest way to get all the friends, and iterate over all of them?
I've tried to figuring out the gem's documentation on cursors, but the fact that friends returned 22 results is throwing me off...
The 20 that you mention is the number of users per page returned by the API. The gem will iterate over all pages, so it will return as many users as possible within the rate limit.
As how to handle the rate limiting, it's up to you. There's an example here that will just put the process to sleep for a while and then retry. Another option is to just quit the app and run it again in a few minutes with something like cron, but it obviously depends on your app.
This is a great question. I ran into the same issue.
I can see how iterating over ALL friends is a feature. However, if you are only interested in the first 20 friends , here's how to not request any more friends:
twitter_client.friends.to_h[ :users ]
Since friends request returns 20, this may not be so useful. If you want friend_ids or follower_ids, the API will return up to 5,000. For many use cases 5,000 is plenty.
twitter_client.friends.to_h[ :ids ]
I want to count the number of visits on my blog? Can someone please suggest the overall method to implement this feature?
It is just an idea. You can add a count_view column in the database into blogs table with default value 0.
And in the show action of BlogsController add the following code
def show
#blog = Blog.where('id = ?', params[:id]).first
#blog.update_column('count_view', #blog.count_view + 1) if #blog.present?
end
You can modify this logic as per your requirement.
You can check the hit counter gem or the impressionist gem.
You could also use an existing (free) analytics solutions if you want to get much more data than the number of times the action was called (please note that if the same user refreshes the browser 5 times, you get 5 hits):
http://www.google.com/analytics/
Using these you can get data like unique visitors, referral URL, locations data, browser, OS, and a lot of different stuff to make informed decisions. There are several other options (paid, free, real time) available as well:
https://mixpanel.com
https://www.kissmetrics.com/
I am building a rails app where there is an image gallery and users will be able to hit the facebook like button for each image they open. Each image can have its seperate page URL. I plan to use the facebook like plugin so that users can like the photos as web-pages. So far I was able to find that if I do https://graph.facebook.com/?id=http://www.mysite.com/ImagePage i get:
{
"id": "http://www.mysite.com/ImagePage",
"shares": 30
}
This leaves me with 2 problems.
The shares property is not exactly the total count of likes, it is only the count of times the link made it to the user's facebook wall. How do I get the number of likes?
Is there an eventHandler or something I can use to know when a user clicks Like? so that I can store that information? I want to store the likes count at my end so that I can show the gallery in the order of descending number of total likes in each day.
I have come across the rails Koala gem but I am not sure if I need to use that for my application yet, as I do not have the need to log in users using facebook login/connect. Please advise if you think I need to do so to do what I mentioned above.
you have to use FQL.
see this example in php, im sure you know how to handle it in ruby:
$fql = 'SELECT url, share_count, like_count, comment_count, total_count
FROM link_stat WHERE url="http://stackoverflow.com"';
$json = file_get_contents('https://api.facebook.com/method/fql.query?format=json&query=' . urlencode($fql));
result looks like:
[
{
"url" : "http://stackoverflow.com",
"share_count" : 1353,
"like_count" : 332,
"comment_count" : 538,
"total_count" : 2223
}
]
For those who don't know how to modify the code seen in the previous answer by yourself, you mostly have two choices:
Use facebook gems available that support running FQL queries (for example: fb_graph)
Sample code:
require 'fb_graph'
array = FbGraph::Query.new(
'SELECT name FROM user WHERE uid = me()'
).fetch(ACCESS_TOKEN)
p array
Use a simple Ruby code that basically just does FQL queries, as seen on this accepted answer (Facebook FQL Query with Ruby)