Can I get all the tweets of a particular user, in which a particular hashtag is used?
I could just get the past 100 tweets then filter them on my end, but it would be much more convenient if Twitter could do the filtering on their end.
Is this possible?
Twitter's Advanced Search lets you construct the query you desire. Unfortunately, search results only seem to be for tweets posted within the past 24 hours or so right now; might just be a temporary limitation, not sure.
In any case, your request URL might look something like this (note the JSON and Atom types):
http://search.twitter.com/search?q=%23HASHTAG+from:USERNAME
http://search.twitter.com/search.json?q=%23HASHTAG+from:USERNAME
http://search.twitter.com/search.atom?q=%23HASHTAG+from:USERNAME
But with the current timeframe limitation, it's not very practical.
I am reading the twitter documentation and they say right now it will include only 6-9 days of tweets. Link https://dev.twitter.com/docs/using-search
To do this search against v1.1 of Twitter's API (which requires OAuth), you'd issue a GET request to the a URL structured like:
https://api.twitter.com/1.1/search/tweets.json?q=from:whitehouse+#sotu
See their Search API's "How to build a query" section. You'll be constrained to tweets over the last 7 days, though.
Related
I've been using the old url api(v1) to get the count of a given url, lately I needed to get also the re-tweets and started searching about that.
this is the exact url I'm using right now:
http://urls.api.twitter.com/1/urls/count.json?url=http://google.com
As I viewed with some reading the v1 api is deprecated but at least it's still working.
I found some questions on the dev page of twitter:
https://dev.twitter.com/discussions/12643
those are a little old questions and have no specific solving to the problem. I mean, the most near solution was using the search api(search/tweets) which could be good but not a exactly replacement for the urls/count method.
Please note that Twitter's search service and, by extension, the
Search API is not meant to be an exhaustive source of Tweets. Not all
Tweets will be indexed or made available via the search interface.
also it has a limit for 100 results at maximum per 'page', even it throws the link to get the next set of objects, thats good but when the search reaches 1 million of results I'll need to get page over page to now how much tweets I got and having to do to much request to the api...
I sought some question over the dev page on twitter suggested using the stream api, I've tried using (statuses/filter) but that don't work very well given a URL as track param(which they said that is the keyword to track).
So, anyone who's been using the old urls/count has found a reliable alternative with the new apiv1.1, especiffically to get the tweets and re-tweets for a given url ?
The official suggestion by Twitter staff is that either the search/tweets endpoint (having just the last 7 days data) or the Streaming API be used (handling yourself the counters, making everything just too complicated for a d*mn counter).
As an extra warning, the old endpoint (http://urls.api.twitter.com/1/urls/count.json?url=YOUR_URL) will stop working on November 20th, and according to this blog post from Twitter there are no plans to replace it with anything in the short term and they are even removing the count from their own buttons.
I am trying to search keywords in twitter through tweepy.
However, I found it seems like that I can not search the tweets one week before, the code below is the main search code.
for searched_tweets in tweepy.Cursor( API.search,
q = "python",
since = 2014-02-03,
until = 2014-02-04,
lang='en' ).items()
I am not sure whether there is any limited or any better way to search by time, thanks for your help!!
:)
Unfortunately, you can't get tweets older than a week with the twitter search API.
Also note that the search results at twitter.com may return historical results while the Search API usually only serves tweets from the past week. - Twitter documentation.
You can get specific tweets older than a week by looking at individual users or using specific post ids (if you have them) but it's not reasonable to index every single tweet ever to be searchable using the API.
If you need a large time range, you can collect them yourself using the streaming API or check out a service that does (see this dev twitter thread for examples).
I'm using the following query:
https://twitter.com/search.json?callback=?&q=from:_adeel&count=5&include_entities=true&include_rts=true
and I would like to see the last 5 tweets from this user. You can see that the user has many more than 2 tweets yet the query only returns 2 tweets. I'm following the twitter search API and I fail to see what I'm doing wrong. I prefer to use the twitter search API because I want to perform the search on the client side without authentication and rate limits.
Do you see what I am doing wrong?
As indicated here in the documentation, the search API is an index of recent tweets, between 6-9 days old. It looks like the other tweets from the user you linked to are just outside this window.
So to summarize - you are not doing anything wrong. This is simply a limitation of the Twitter API.
I have seen numerous companies doing like Twitter lotteries where users got to retweet their tweet and then one of retweeters will get the prize or whatever.
I was wondering now how do they get the list of all retweeters or pick the winner?
I checked Twitter API and found only this thing: GET statuses/retweets/:id but it returns maximum of 100 retweeters. Is that the only way?
It looks likes there's a couple services out there doing almost exactly this. A quick google pulls up http://onekontest.com/ and there's a few other Twitter contest services, but they all seem to be different levels of broken since they haven't kept up with changes to the API.
As far as the Twitter API itself is concerned, if you were expecting more than 100 responses, I think using GET statuses/mentions makes the most sense. That API call returns any mentions of a user, and you can pass the flag include_rts to include any retweets of your tweets. Then, if you wanted to list RTs of a specific tweet, you could check the in_reply_to_status_id field in the returned data to see if it matches the original tweet ID. This API call only returns the last 800 status, 200 at a time, so if you expect a bunch of data, you would need to poll the API repeatedly over time to get all the tweets. I imagine services like favstar are doing exactly this, just on a larger scale.
If you're actually looking for code to do something like this, I wrote a sinatra app called twitter-rss-digest which handles querying Twitter over time to track different sorts of queries. It's pretty rough, and doesn't quite handle this specifically, but it might point you in the right direction if you want to code something.
The Twitter API has an endpoint that returns up to 100 retweeter IDs for a given tweet.
Note that for historical reasons, that endpoint only returns up to 100 results and the documentation about the cursor is useless. Twitter refused to fix the situation.
2013 note
That said, the first thread on the Developers site that surfaced in a quick google has #episod, a Twitter employee saying:
You can't likely get to all of them. The data is spread out. Your best bet is to use the REST API to determine many of the users who performed the retweet, but the data will still be constrained.
It's easiest to track retweets as they happen rather than try to find them from the past. Use the Streaming API for that.
I like muffinista's method, but I think if you want a 100% complete list of retweets, simply enable the retweet email notifications and write a script that polls the email box for those matching the subject "retweeted one of your Tweets!" and put the data into a table. Do this right from the start.
The site https://twren.ch/ enlists all the retweeters for a given tweet (note that it only enlists retweeters who are direct followers of the source tweeter.) Nevertheless its probably the only public source available.
I want to gather samplings of all tweets from the past year. Being able to request tweets from a specific date would be great, but I'll take what I can get.
I do not want to find tweets by a specific user or containing a specific term, just a sampling of all tweets. The Twitter search API claims that a query term is optional, but if I try an empty query like
http://search.twitter.com/search.atom
as opposed to giving a search term,
http://search.twitter.com/search.atom?q=twitter
the response is
<hash>
<error>
You must enter a query.
</error>
</hash>
If the API really doesn't provide any functionality for this type of query, how can I hack around it? Are tweet ids roughly sequential by date and can I somehow use this info to grab bunches of tweets centered around an id of a tweet whose date I know?
You are referencing the obsolete documentation. If you read the current version you will find that a query is required.
You should also know that the Search API only provides results going back about two weeks. You might be able to find historical data from sites like infochimps.
Not useful for historical data, but in case someone stumbles across this question looking for a sampling of all current tweets, you want the streaming API. (This is my first foray into Twitter and I hadn't noticed it. I only saw the public timeline method in the normal API.)