My test tweets are not showing up in the twitter search. I have asked several other people to test this as well and only a few show up.
I am trying to track a campaign on the number of people who tweet a certain hashtag. How am I supposed to track it accurately if the search does not give exact results?
Has anyone else experienced this problem?
I read where the API isn't extremely accurate. It only returns results after a crawl, and may not have crawled the entire database when you're searching. The API also doesn't hold on to tweets longer than a few days old.
Related
i'm University students of South korea
I'm developing analysis application using bigdata of twitter with my advisor professor. So i'm gathering tweets contains specific keyword(relevant word of crime) at period. I use 'streaming api' and 'search api' now. I have seen that using search api and streaming api result is return tweets of only one week.
I should be get the old data that have keyword of crime and since 2006 until 2016
do you have any idea?
Sadly you can't get tweets from that time range.
From the documentation:
The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets.
So, you can only get recent tweets from the search API. Be careful too with the data beacuse it's about relevance not completeness, from the same documentation:
Before getting involved, it’s important to know that the Search API is focused on relevance and not completeness. This means that some Tweets and users may be missing from search results. If you want to match for completeness you should consider using a Streaming API instead.
If you really need older tweets you will have to get them from other sources like Gnip. Otherwise you will have to approach differently your problem.
If you have the names (or id's) of all the users that you want to get info you could get the timelines from each user getting up to 3200 tweets.
I am trying to search keywords in twitter through tweepy.
However, I found it seems like that I can not search the tweets one week before, the code below is the main search code.
for searched_tweets in tweepy.Cursor( API.search,
q = "python",
since = 2014-02-03,
until = 2014-02-04,
lang='en' ).items()
I am not sure whether there is any limited or any better way to search by time, thanks for your help!!
:)
Unfortunately, you can't get tweets older than a week with the twitter search API.
Also note that the search results at twitter.com may return historical results while the Search API usually only serves tweets from the past week. - Twitter documentation.
You can get specific tweets older than a week by looking at individual users or using specific post ids (if you have them) but it's not reasonable to index every single tweet ever to be searchable using the API.
If you need a large time range, you can collect them yourself using the streaming API or check out a service that does (see this dev twitter thread for examples).
I have list of users
['foo','bar']
I want to search whether they have checked in somewhere or not (using 4sq api)..
So basically, all I am looking for is that whether their tweets contain "\4sq.com\" or not?
I get very confused looking into their api?
Bonus points if the steps can be implemented in python.
Thanks
You have two options for checking a user's tweets:
One option is to look through the tweets in their User Timeline (accessible in Tweepy through api.user_timeline). However, you may have to search through a lot of unrelated Tweets before you come across the one your looking for. Given how many tweets some users have, you might want to only look thorough tweets more recent than a certain date (you can look at the created_at attribute of returned tweets).
The other option is to use the Search API (accessible in Tweepy though 'api.search'). This has the advantage of allowing you specify a search query, giving you relevant tweets. However, you will need to search through the tweets until you find one by the user you're looking for. Again, you might want to limit the date range of tweets that you search through.
I am developing an application to guess locations of tornadoes by analyzing twitter data. For this, I would first need to train a neural network on some manually annotated tweets. I am trying to get tweets from last year which have the word 'tornado' in them. This is my code below :-
Query query = new Query("tornado");
query.setRpp(100);
query.setSince("2010-11-01");
query.setUntil("2011-01-13");
QueryResult queryResult = instance.search(query);
tweetList = queryResult.getTweets();
I am able to retrieve tweets from periods closer to now such as last week and such, but am unable to get any results for periods such as the one listed above. Any clues, suggestions would be help. Thanks in advance.
I just found out the reason through a different medium, thought i'd share the answer in case there are other people with the same issue.
It turns out that the twitter search api does not return tweets older than around a week and also, depending on the server load, at times this could be as low as 24 hours ! Hence, any 3rd party libraries (such as twitter4j) which have a wrapper for the twitter search api will behave similarly.
The best way to go about this would be to use third party search and indexing sites such as snapbird, topsy, etc..
I want to gather samplings of all tweets from the past year. Being able to request tweets from a specific date would be great, but I'll take what I can get.
I do not want to find tweets by a specific user or containing a specific term, just a sampling of all tweets. The Twitter search API claims that a query term is optional, but if I try an empty query like
http://search.twitter.com/search.atom
as opposed to giving a search term,
http://search.twitter.com/search.atom?q=twitter
the response is
<hash>
<error>
You must enter a query.
</error>
</hash>
If the API really doesn't provide any functionality for this type of query, how can I hack around it? Are tweet ids roughly sequential by date and can I somehow use this info to grab bunches of tweets centered around an id of a tweet whose date I know?
You are referencing the obsolete documentation. If you read the current version you will find that a query is required.
You should also know that the Search API only provides results going back about two weeks. You might be able to find historical data from sites like infochimps.
Not useful for historical data, but in case someone stumbles across this question looking for a sampling of all current tweets, you want the streaming API. (This is my first foray into Twitter and I hadn't noticed it. I only saw the public timeline method in the normal API.)