I'm trying to create a widget on my blog that grabs about 20 twitter accounts and displays the last two posts from each account. At the moment I have a jquery based version that runs a for loop for each account. Naturally I bumped up against the 150 requests per hours maximum pretty fast. My question is two fold:
Is there a more efficient way to grab this info? Say opening a twitter account, follow all the users I need and then just reference the friends aspect of my account?
Is there a way to cache the JSON response so I don't bump into the limit? Or do I have to write the entire results to a file and reference that once the limit has been hit?
I should also mention the platform I'm working on is Wordpress/LAMP but I don't care what the technology end up being, I'm open to flash, etc to make this work.
Thanks all
To guarantee showing two posts from each account, you'd probably need to pull each feed down separately. If one author wrote 100 tweets then no one else's tweets would be easily reached within a combined feed.
Your best bet is to get some PHP to download the tweets for you and cache them temporarily on the server, rather than using jquery and AJAX on the client side. A fairly simple Wordpress plugin would do it if you can do PHP. Alex King wrote a Twitter WordPress plugin which can pull down one account, perhaps you could modify it to subscribe to more.
Yeah, your suggestion seems like the most optimal way. Create an extra Twitter account and use the friends timeline to aggregate this data. Keep all the large-scale processing on Twitter's end.
Just be sure to cache your results server-side for about a minute or so.
It is possible to get the latest tweets back from multiple users from one request using Twitter's search API. The q parameter can take multiple from:user values. Below is a quick example, using jQuery:
var query = escape("from:biz OR from:twitter OR from:bob"),
url = "http://search.twitter.com/search.json?callback=?&q=" + query;
$.getJSON(url, function(data){
$.each(data.results, function(i, tweet){
$('body').append("<p>" + tweet.text + " by " + tweet.from_user_name + "</p>");
});
});
Related
I've got a bunch of free online HTML, CSS, and JS tutorials under my belt and I want to try using them to make a browser extension. But I want to make sure that the data I want to use is actually accessible before getting started.
My goal is to make a browser extension for twitter.com that shows the number of impressions of any tweet next to the likes, retweets, and replies. My basic idea is to get the status URL of any given tweet, poll the Twitter API for the number of impressions of that tweet, store that in a variable, and then use CSS to display a little eye icon and the number stored in the impressions variable.
I know that I can find the number of impressions of all of my tweets, both through Twitter Analytics, and also just going to my profile page and clicking the little bar chart icon next to views, retweets, etc. But I'm not clear on whether I can do that for other people's tweets via Twitter's API or anything else. Can you?
For the record, I'm not too concerned about the varying definition of "impression," since it will be consistently applied across all tweets and I'm mostly interested in giving users a comparison between tweets. This is part of a research project to see how this might change how people engage with social media if they know how many views a given post has. If there's a simpler way to go about that using existing platforms, I'm open to suggestions.
Thanks for the advice!
No, impressions data is private. If you are authenticated to the Twitter API then you can use the new Twitter Developer Labs Tweets API to get private metrics like impressions, but you cannot get that for other people's Tweets. Also, the Twitter API does not support CORS, so I don't think you'll be successful trying to use it from a browser extension.
I am new to Twitter and need some tips.
I need to display tweet feed from multiple users on some webpage.
The first thing I stumbled upon is Embedded Timelines. It allows to display tweets from list of users but the gotcha is that those lists should be maintained on Twitter-side (i.e. I cannot specify #qwe and #asd only on my side and get timeline without adding those users into list on Twitter-side).
The thing is that list of users that should be included into timeline is dynamic and managing those lists through Twitter API will probably be painful. Not to mention that my website will probably generate tons of those lists and I feel that I will violate some api quotas sooner or later.
So, my question is - am I stuck with using Embedded Timelines that refer some user list on Twitter-side and managing those lists through, say Twitter REST api, or there is a simplier way to do what I want?
It's pretty simple to display tweets for multiple users.
Links to start with
This post explains some of the search queries you can make
This post is a simple library to make requests to the twitter API that 'just works'
Your Query
Okay, so you want multiple users. The endpoint you're looking at using is the search/tweets one: https://api.twitter.com/1.1/search/tweets.json.
The query string uses :from and you can interpolate multiple froms with AND/OR.
An example query for the GET request:
?q=from:user1+OR+from:user2
Read more about the search API queries here.
Your "over-the-quote" issue
This is something you're going to need to figure out yourself - depending on the number of requests you expect to make, and the twitter imposed limits, maybe some sort of caching or saving information when you hit your limit, and only pull back from the cache whilst you're hitting your limit..
I am building a Twitter app and I'll be pulling a big amount of data from the user's timeline. For speed, I need to query the timeline in parallel. My aim is to pull 1000 of user's tweets from the API, but the upper limit of number of tweets per request is set to 200 by the Twitter API. Pagination works by specifying the last (oldest) tweet's ID from the previous request, so I need to know the result of the previous API call to make the next call. This method is not parallelizable. Is there any alternative method for getting the user timeline from the Twitter API where I can make parallel requests (there is the page property, but is deprecated and will be nonfunctional in the near future).
What you have to remember, is that Twitter have a difficult relationship with external developers. Using their API for anything interesting like this is simply not allowed by them.
What you need is access to the Firehose.
However, even if you're willing to pay a million dollars a year - Twitter aren't interested.
You could try getting it from a third party like Gnip but - again - likely to be expensive.
So, essentially, you can't. Twitter just aren't interested in amateur developers doing anything innovative with their platform. Sorry.
I am developing an application to guess locations of tornadoes by analyzing twitter data. For this, I would first need to train a neural network on some manually annotated tweets. I am trying to get tweets from last year which have the word 'tornado' in them. This is my code below :-
Query query = new Query("tornado");
query.setRpp(100);
query.setSince("2010-11-01");
query.setUntil("2011-01-13");
QueryResult queryResult = instance.search(query);
tweetList = queryResult.getTweets();
I am able to retrieve tweets from periods closer to now such as last week and such, but am unable to get any results for periods such as the one listed above. Any clues, suggestions would be help. Thanks in advance.
I just found out the reason through a different medium, thought i'd share the answer in case there are other people with the same issue.
It turns out that the twitter search api does not return tweets older than around a week and also, depending on the server load, at times this could be as low as 24 hours ! Hence, any 3rd party libraries (such as twitter4j) which have a wrapper for the twitter search api will behave similarly.
The best way to go about this would be to use third party search and indexing sites such as snapbird, topsy, etc..
is there any service from where we can download tweets?
UPDATE!!!
Googling for sometime gave me this result
a.) http://snap.stanford.edu/data/twitter7.html
b.) http://140kit.com/datasets
Yes, there is. It's called the Twitter API.
As we have access to limited tweets by Twitter-API, we should make use of third-party resellers like Topsy for just the past data, GNIP just for streaming data, or DataSift for both streaming data as well as past data.
You might also want to check the following sites:
http://www.infochimps.com/collections/twitter-census
http://www.tweetarchivist.com/
Twitter API allows provides partial results, it gives you the last 100 or even 500 tweets fo every search. If you need to keep tweets long term, twitter API shows its limits.
I had same need as you apparently hae and I developed a tool that queries twitter API periodically and stores search results on a Wordpress database.
I called the tools twittcorder and you can find a live demo on twittcorder.com
I hope this helps.
These other data sources are probably shared against the Twitter TOS. I wouldn't want to invest my time and effort building something on datasets that are non-repeatable. The Twitter Streaming API allows collection of a sample of Tweets.
There's also Gnip: http://gnip.com/.
Sysomos is there for complete data analysis including twitter, faecbook and various boards and forums