Whenever I download tweets using flume all the downloaded tweets are the tweets that are tweeted after the query is run. Is there any way to download the tweets which are already tweeted before running the query??
for example if I run a query at 9.00am then the tweets that are created after 9am will only be downloaded.How can I download the tweets that are created before 9am?
Flume uses Twitter streaming API, using streaming API you can't get old tweets.
Related
I want to crawl the tweets from twitter for sentimental analysis and want to know if by using my own crawler and without OathAuthentification i.e by registering my app on twitter can I crawl twitter
How can I download all (or most number) of someone's twitter timeline?
Is there any website can do this, or maybe some application or excel sheet.
I am not that good with API, so anything simple will be good.
You will have to use the Twitter Timeline REST API and iterate to get up to 3200 tweets from his timeline.
Documentation : https://dev.twitter.com/rest/reference/get/statuses/user_timeline
If you are using C# I can provide an example using Tweetinvi.
I am trying to access live tweets of a user whenever he tweets it. So, all I want is something that continuously monitors a user account and whenever he tweets something I have to capture it. All the tweets are random so I cannot use any filters.
For any security reasons, if I cannot access other's tweets can I do it on my own account?
With tweepy you can connect to the REST API or the Streaming API.
Using the Streaming API you can use the filter endpoint to select the users you wish to follow with that streaming connection and you will receive updates as they get published.
Twitter's documentation: https://dev.twitter.com/docs/api/1.1/post/statuses/filter, tweepy's code: https://github.com/tweepy/tweepy/blob/master/tweepy/streaming.py
Tweepy's documentation doesn't give examples on the Streaming functions but you can find sample code searching at GitHub or StackOverflow for "tweepy filter follow".
Tweepy talks to the Twitter REST API, and the REST API doesn't have any way to react to someone posting a tweet.
HOWEVER...
You could certainly write an application that retrieves the tweets of a partcular user and looks for any tweets that weren't there the last time you checked.
You'd want to be cautious about how often you check so you don't run afoul of the API rate limits.
We have a manual process to download the performance report from youtube cms application. This report is a csv or zip that is generated weekly and gives the "view" details for every video owned by the user at a daily level.
Is there an API to download this report ? I looked at the youtube analytics api and it looks like I have to get the data for each video individually. We have a lot of videos and this will be over the quota. Is there a way to get the weekly performance report through some api ?
The current YouTube Analytics API isn't well suited for performing bulk reporting for any significant volume of videos.
There's an open feature request that you can "Star" to be notified when there's an alternative approach available. I'm afraid that I don't have any timelines to share, but it's being actively worked on.
As Jeff pointed out before, you won't be able to download a performance report from Youtube CMS using Youtube Analytics API.
You can use mechanize module in python to log in to your youtube cms account and navigate to that download URL.
browser = mechanize.Browser()
browser.open('http://cms.youtube.com')
browser.select_form(nr = 0)
browser.form['Email'] = email
browser.form['Passwd'] = password
browser.submit()
browser.open(dashboard_url) //url that you get redirected to when you login
and then you can search for reports link. then click on that. then search for the report you want and download it.
more info here:
http://www.pythonforbeginners.com/cheatsheet/python-mechanize-cheat-sheet/
i was able to download reports that i want using this module in python.
I'm currently using Twitter's RSS API (example) to retrieve people's tweets for an app. However, once version 1.1 of their API comes out, I won't be able to do this anymore. What's a simple replacement I can use for this?
Unfortunately the answer is: "There is none."
Twitter continues to lock down their platform, as they have since the switch to OAuth. If you want to add content to Twitter, you can do that easily. If you want to get content out of Twitter, well, you'll have to conform to Twitter's requirements.
RSS was an easy way of getting content out of Twitter, and so they killed it. They'll tell you that it wasn't really used, but that's a load of bull and they know it, but it sounds good and most people will simply swallow it.
There is a workaround that you can use until March 2013:
https://api.twitter.com/1/statuses/user_timeline/UserName.rss
Replace 'UserName' with the actual twitter username you want.
But after sometime in March/April 2013, that will cease to work and you'll have to use OAuth and json to parse Twitter feeds.
Use Twitter RSS - Google Apps Script from github to convert Twitter API 1.1 to RSS. Using this option you can get search results, users' timelines, users’ favorites or even Twitter Lists.
I've put together two mit licensed php scripts and instructions so you can fetch and cache twitter user timelines as RSS..
The most important is to cache the tweets since you are only allowed to call the API 15 times per 15 minutes..
Have a look at twitterrss.net
I've been looking for an answer to this for a while, basically as I wanted to provide a replacement for the Twitter triggers in IFTTT. When IFTTT removed them we could use Twitter's RSS feeds. However now, with v1 of their API retired, you have to authenticate any requests and they no longer off RSS feeds. It's a shame because it's closing the door for a lot of newbie developers out there.
I'd definitely recommend checking out twitter-rss.com in the answer above, but you might want to check out a solution I've come up with. It is called Twools and it's almost ready.
Twools started off as a way to get back RSS feeds from Twitter. Twitter
retired version 1 of their API in June 2013 as well as retiring RSS feeds. As
well as not having access to RSS feeds, you need to be authenticated to use
the API which
raises the bar for Twitter users and developers. It has been my intention to
make this as easy as possible for you install, however you do need to install
this on your own website (PHP is required). It is just a case of editing a few
lines in a file (with your Twitter app details) and uploading to your website.
Twools offers a number of RSS feeds from the Twitter API:
Your home timeline
User timeline (yours or another user's)
Your mentions
Your favorites
List timeline (from a list that you own)
Direct messages received (disabled by default)
Direct messages Sent (disabled by default)
Twitter Search
New followers
New friends
You can filter the results from the Twitter API by only outputting tweets
containing certain keywords, hashtags or screen names or even by the app used
to send the Tweet. As well as that you can filter based on the user (currently
their location and language). Finally, you can modify the data outputted in
the RSS feed by removing URLs, screen names, hashtags etc or share your recipe
with other folders. As well as an RSS feed generator, Twools has a basic
direct message spam filter (experimental- use at your own risk!) and a simple
social media management tool (basically an HTML version of the RSS filters).
You can also keep an eye on your API usage on the Status page.
With Twools and your app you could
Your tweets when you mention a specific hashtag
Your Twitter Faves:
Get new followers from the a certain country or location
Get tweets when someone mentions Starbucks negatively in a Tweet in the UK
Get tweets when someone tweets to an article on your website
For more information, see http://twools.it/