We are attempting to use the Twitter Search Widget in our website: Twitter widget
However, it only seems to display the most recent tweets.
For example, if using the following search query, we only get back 4 results posted in the last 25 hours, but none older: from:CorkBoston since:2010-01-01
Is this a limitation? I don't see it in any of the documentation
It is their limitation:
The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets.
You cannot use the Search API to find Tweets older than about a week.
https://dev.twitter.com/docs/using-search
Related
i'm University students of South korea
I'm developing analysis application using bigdata of twitter with my advisor professor. So i'm gathering tweets contains specific keyword(relevant word of crime) at period. I use 'streaming api' and 'search api' now. I have seen that using search api and streaming api result is return tweets of only one week.
I should be get the old data that have keyword of crime and since 2006 until 2016
do you have any idea?
Sadly you can't get tweets from that time range.
From the documentation:
The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets.
So, you can only get recent tweets from the search API. Be careful too with the data beacuse it's about relevance not completeness, from the same documentation:
Before getting involved, it’s important to know that the Search API is focused on relevance and not completeness. This means that some Tweets and users may be missing from search results. If you want to match for completeness you should consider using a Streaming API instead.
If you really need older tweets you will have to get them from other sources like Gnip. Otherwise you will have to approach differently your problem.
If you have the names (or id's) of all the users that you want to get info you could get the timelines from each user getting up to 3200 tweets.
For example how could search for tweets using the #oldtweets sent on 5-29-16, from 8pm to 9pm.
According to the The Twitter Search API documentation, the query you want is not possible: https://dev.twitter.com/rest/public/search
The Twitter Search API searches against a sampling of recent Tweets
published in the past 7 days.
Beyond the last 7 days, what you want to achieve can only be done through manually searching an account on Twitter.
Another alternative would be to use https://webrecorder.io/
Scroll to the timeframe you want to record or you can attempt to capture the entire feed. Note the auto scrolling option as well.
I have a Twitter feed I'm using for a site, and in theory it should work perfectly, but for some reason
http://search.twitter.com/search.atom?q=from:#goinnative&rpp=10
only loads 3 tweets (appears to be tweets from the past week) instead of the 10 most recent tweets. Is there some way to use the same feed to call a longer date range?
Got it, used this feed instead.
http://api.twitter.com/1/statuses/user_timeline.rss?screen_name=goinnative
For an academic usage, I would like to analyze about three months of tweets. However, it seems the official Twitter search API doesn't provide tweets older than one week.
I've tried to write a self crawler, however, given a search keyword, Twitter page will not show tweets older than about one week.
Is there any trick that I can get older tweets? Or my best bet is to hit the API once a week and do it for the following three months?
From the Twitter API documentation regarding limitations:
- The Search API is not complete index of all Tweets, but instead an index of recent Tweets.
- At the moment that index includes between 6-9 days of Tweets.
- You cannot use the Search API to find Tweets older than about a week.
So, yes, if you need to collect a certain span of time, it will require multiple queries, as you suggested.
(You should also read this answer: retrieving tweets from specific user older than 7 days)
There are also currently two commercial companies that have access to the Twitter firehose and can provide this data (they are called "licensed re-syndicators"):
Gnip - offers 30 days of Twitter data
DataSift - up to two years of Twitter data
I'm using the following query:
https://twitter.com/search.json?callback=?&q=from:_adeel&count=5&include_entities=true&include_rts=true
and I would like to see the last 5 tweets from this user. You can see that the user has many more than 2 tweets yet the query only returns 2 tweets. I'm following the twitter search API and I fail to see what I'm doing wrong. I prefer to use the twitter search API because I want to perform the search on the client side without authentication and rate limits.
Do you see what I am doing wrong?
As indicated here in the documentation, the search API is an index of recent tweets, between 6-9 days old. It looks like the other tweets from the user you linked to are just outside this window.
So to summarize - you are not doing anything wrong. This is simply a limitation of the Twitter API.