For example how could search for tweets using the #oldtweets sent on 5-29-16, from 8pm to 9pm.
According to the The Twitter Search API documentation, the query you want is not possible: https://dev.twitter.com/rest/public/search
The Twitter Search API searches against a sampling of recent Tweets
published in the past 7 days.
Beyond the last 7 days, what you want to achieve can only be done through manually searching an account on Twitter.
Another alternative would be to use https://webrecorder.io/
Scroll to the timeframe you want to record or you can attempt to capture the entire feed. Note the auto scrolling option as well.
Related
i'm University students of South korea
I'm developing analysis application using bigdata of twitter with my advisor professor. So i'm gathering tweets contains specific keyword(relevant word of crime) at period. I use 'streaming api' and 'search api' now. I have seen that using search api and streaming api result is return tweets of only one week.
I should be get the old data that have keyword of crime and since 2006 until 2016
do you have any idea?
Sadly you can't get tweets from that time range.
From the documentation:
The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets.
So, you can only get recent tweets from the search API. Be careful too with the data beacuse it's about relevance not completeness, from the same documentation:
Before getting involved, it’s important to know that the Search API is focused on relevance and not completeness. This means that some Tweets and users may be missing from search results. If you want to match for completeness you should consider using a Streaming API instead.
If you really need older tweets you will have to get them from other sources like Gnip. Otherwise you will have to approach differently your problem.
If you have the names (or id's) of all the users that you want to get info you could get the timelines from each user getting up to 3200 tweets.
I am trying to search keywords in twitter through tweepy.
However, I found it seems like that I can not search the tweets one week before, the code below is the main search code.
for searched_tweets in tweepy.Cursor( API.search,
q = "python",
since = 2014-02-03,
until = 2014-02-04,
lang='en' ).items()
I am not sure whether there is any limited or any better way to search by time, thanks for your help!!
:)
Unfortunately, you can't get tweets older than a week with the twitter search API.
Also note that the search results at twitter.com may return historical results while the Search API usually only serves tweets from the past week. - Twitter documentation.
You can get specific tweets older than a week by looking at individual users or using specific post ids (if you have them) but it's not reasonable to index every single tweet ever to be searchable using the API.
If you need a large time range, you can collect them yourself using the streaming API or check out a service that does (see this dev twitter thread for examples).
I have a Twitter feed I'm using for a site, and in theory it should work perfectly, but for some reason
http://search.twitter.com/search.atom?q=from:#goinnative&rpp=10
only loads 3 tweets (appears to be tweets from the past week) instead of the 10 most recent tweets. Is there some way to use the same feed to call a longer date range?
Got it, used this feed instead.
http://api.twitter.com/1/statuses/user_timeline.rss?screen_name=goinnative
For an academic usage, I would like to analyze about three months of tweets. However, it seems the official Twitter search API doesn't provide tweets older than one week.
I've tried to write a self crawler, however, given a search keyword, Twitter page will not show tweets older than about one week.
Is there any trick that I can get older tweets? Or my best bet is to hit the API once a week and do it for the following three months?
From the Twitter API documentation regarding limitations:
- The Search API is not complete index of all Tweets, but instead an index of recent Tweets.
- At the moment that index includes between 6-9 days of Tweets.
- You cannot use the Search API to find Tweets older than about a week.
So, yes, if you need to collect a certain span of time, it will require multiple queries, as you suggested.
(You should also read this answer: retrieving tweets from specific user older than 7 days)
There are also currently two commercial companies that have access to the Twitter firehose and can provide this data (they are called "licensed re-syndicators"):
Gnip - offers 30 days of Twitter data
DataSift - up to two years of Twitter data
Hi all
I am new to twitter API.. So my question may be a trivial one..
I am developing a new twitter widget, in which I want to get the first tweet for a given hashtag on a particular date.
say first tweet for #xyz hashtag tweeted on 12-08-10.
so far through googling and api doc I found rpp * page max = 1500.. if totaltweet for a given day is 2000 how can I retrieve last 500 tweets..
I have a few questions?
1. Is this for any hashtag that a user might request on the fly, or a set of hashtags that generally remain the same?
If it is a set of tags that stays the same for a while, roughly how many tags are there?
When you say a particular date, how far back in time are you going?
There are two standard ways to get tweets for a hashtag. You can use search, which goes back around 7 days or 1,500 tweets. For an active tag, that may only be a day or two. You can use the streaming API, but that doesn't go back, it just starts tracking from the moment you start making the request. The streaming API can track up to 400 hashtags without getting approval for a higher rate. The search API is limited to 10 to 20 hashtags per query.
If you answer questions 1-3, I can suggest a path to getting what you want.