Good evening, StackOverflow!
I am working with twitter API to retrieve tweets from the twitter. I need to receive tweets by a time range, for example, all tweets from #someUser for the last week, or last 10 days... or from March to April.
I am looking at https://dev.twitter.com/rest/reference/get/statuses/user_timeline documentation, and it doesn't really specify how to use the date to narrow down the list of receiving tweets. I mean, I can use since_id or max_id, but it is not really what I'm looking for.
I mean, I can receive tweets by one using max_id and since_id, check the date, and see if it falls into my time frame, and continue until I get to the edge. Anyway, it is not a good practice.
So, is there a way or maybe I missed something in the documentation?
I really would appreciate any help (just a tip would work).
Related
Let's say I need to obtain the text (post and all comments) from the sub-reddits that were most popular during December 2014 and January 2015. How can that be done using PRAW? I was wondering, just like Twitter doesn't let us crawl tweets which are more than a week old, is there any such time constraint in place in case of Reddit?
PRAW has a limit of 1,000 for how far back you can get threads from a subreddit based on score, newness, "hotness" (I'm not sure if that one still works) or "controversy" (I haven't tried that one in a while, as well). Unfortunately, the most popular subreddits have had too many posts to look that far back.
You can try using search() http://praw.readthedocs.org/en/latest/pages/code_overview.html#praw.init.UnauthenticatedReddit.search to get the values you want, but it requires a bit more work.
TunnelBear offers 1 GB free every month when you tweet about them. I am trying to implement something like this in my web app.
This is how I intend to go about it.
My system will let anyone tweet about my business with some predefined message like "Use Awesome_business to increase profit. Awesome_business is the best!"
A cron job will regularly (every 5 min) fetch the last 10 tweets of all connected twitter accounts
Analyse them to see if they contains keywords like "use", "Awesome_business", "best", "increase", "profit"
Offer discount for every account which tweets will contains keywords listed above.
Does anyone have a better idea about how to go about this?
You could try leveraging Twitter's Streaming API: https://dev.twitter.com/docs/streaming-apis
So you can open a connection and you'll receive tweets as they occur, instead of having to periodically check (plus if there are more than 10 tweets in the last 5 min then you just lost those extras).
Also twitter's search API doesn't allow for specific times, you can only give it a day, so the best you could do is pull all tweets from the last day, and then manually parse the data to find tweets in the specific amount of time from your last poll.
Twitter's API also let's you do hashtag filtering, so you can just grab all tweets with a hashtag relevant to your business, and then reward those people specifically? would make it easier on you so you don't have to write any parsing logic on your end.
I want to gather samplings of all tweets from the past year. Being able to request tweets from a specific date would be great, but I'll take what I can get.
I do not want to find tweets by a specific user or containing a specific term, just a sampling of all tweets. The Twitter search API claims that a query term is optional, but if I try an empty query like
http://search.twitter.com/search.atom
as opposed to giving a search term,
http://search.twitter.com/search.atom?q=twitter
the response is
<hash>
<error>
You must enter a query.
</error>
</hash>
If the API really doesn't provide any functionality for this type of query, how can I hack around it? Are tweet ids roughly sequential by date and can I somehow use this info to grab bunches of tweets centered around an id of a tweet whose date I know?
You are referencing the obsolete documentation. If you read the current version you will find that a query is required.
You should also know that the Search API only provides results going back about two weeks. You might be able to find historical data from sites like infochimps.
Not useful for historical data, but in case someone stumbles across this question looking for a sampling of all current tweets, you want the streaming API. (This is my first foray into Twitter and I hadn't noticed it. I only saw the public timeline method in the normal API.)
Hi all
I am new to twitter API.. So my question may be a trivial one..
I am developing a new twitter widget, in which I want to get the first tweet for a given hashtag on a particular date.
say first tweet for #xyz hashtag tweeted on 12-08-10.
so far through googling and api doc I found rpp * page max = 1500.. if totaltweet for a given day is 2000 how can I retrieve last 500 tweets..
I have a few questions?
1. Is this for any hashtag that a user might request on the fly, or a set of hashtags that generally remain the same?
If it is a set of tags that stays the same for a while, roughly how many tags are there?
When you say a particular date, how far back in time are you going?
There are two standard ways to get tweets for a hashtag. You can use search, which goes back around 7 days or 1,500 tweets. For an active tag, that may only be a day or two. You can use the streaming API, but that doesn't go back, it just starts tracking from the moment you start making the request. The streaming API can track up to 400 hashtags without getting approval for a higher rate. The search API is limited to 10 to 20 hashtags per query.
If you answer questions 1-3, I can suggest a path to getting what you want.
Can I get all the tweets of a particular user, in which a particular hashtag is used?
I could just get the past 100 tweets then filter them on my end, but it would be much more convenient if Twitter could do the filtering on their end.
Is this possible?
Twitter's Advanced Search lets you construct the query you desire. Unfortunately, search results only seem to be for tweets posted within the past 24 hours or so right now; might just be a temporary limitation, not sure.
In any case, your request URL might look something like this (note the JSON and Atom types):
http://search.twitter.com/search?q=%23HASHTAG+from:USERNAME
http://search.twitter.com/search.json?q=%23HASHTAG+from:USERNAME
http://search.twitter.com/search.atom?q=%23HASHTAG+from:USERNAME
But with the current timeframe limitation, it's not very practical.
I am reading the twitter documentation and they say right now it will include only 6-9 days of tweets. Link https://dev.twitter.com/docs/using-search
To do this search against v1.1 of Twitter's API (which requires OAuth), you'd issue a GET request to the a URL structured like:
https://api.twitter.com/1.1/search/tweets.json?q=from:whitehouse+#sotu
See their Search API's "How to build a query" section. You'll be constrained to tweets over the last 7 days, though.