Ive searched through stack, but answers are dated. I was wondering if anyone knows what it is to crawl a topic like security. How do I do this by using Twitter? Do I just follow people who tweet about this topic, re-tweet and tweet new things, or is there an exact way of doing this? I then need to make statistical analysis on the data I gather.
You can use Puppeteer to crawl twitter data.
Checkout their github repository here.
This is a repository that crawls twitter data using Puppeteer .
How about using twitter search api (https://dev.twitter.com/docs/api/1.1/get/search/tweets)
You need to create an app first(or better say register an app) on dev.twitter.com and use search api to query for tweets that contain security (assuming I understood your crawling in the right way). Once you have your tweets you can do statistical analysis on the gathered data.
I use twitteR package on R to crawl twitter data (https://github.com/geoffjentry/twitteR) . It includes simple and useful functions to get twitter data.
Related
I would like to know if there is a java library or API , that can extract tweets of my interest such as i want to know which tweets have reported about the strike in upcoming days in Islamabad city or bomb blast recently occured in my city . etc. I know there are other libraries out there but they only tell about whether the specific tweet is positive , negative or neutral. Thanks
Every library for integrating twitter tweets to your application is based on the twitter apis.
For your specific example, you can try Search API.
The process is really simple: just try from here your keywords to determine which keywords best suit your need and then use the actual API like this(returns tweets with keyword:Islamabad) to return the tweets you need in a JSON format.
NOTE:
Version 1.1 of the APIS uses an OAuth Authentication(I have not tried it yet so I can not provide more details :( ).
Now for Java libraries(frameworks) that simplify this process, the only one I know is Spring Social. But in case you are not familiar with Spring framework in general, the best thing is to just read the Json url generated by the api and unmarshall it to get your results.
I'm trying to develop some code in order to get all the tweets that were generated with certain hashtags, then parse them and finally analyse them. I believe I've already thought and solve the last two parts of this but I'm having some trouble with the first one. I've already read the Twitter Search API documentation but I haven't realised yet how to do this. Can anyone help me?
If you want to retrieve the tweets sent recently, you should use the search/tweets endpoint of twitter' REST API, and mention the hashtag inside q parameter
In case you want to listen to tweets containing the hashtag and receive them in real time, then twitter's streaming API is what you should use (statuses/filter endPoint).
Have a look at the documentation on twitter's website, there's also plenty of information on how to do this all around the web.
I would like to apologize first if the question is a total newbie question, but I really am a total newbie on this.
I'm a student and I recently have joined a project that involves studying (mining) tweets. The project head asked me to use the Twitter API to extract tweets. What exactly is Twitter API and how can I use it? What do I need to know to start using it?
Twitter allows you to interact with its data ie tweets & several attributes about tweets using Twitter APIs. You'd need to know a server side scripting language like php, python or ruby to make requests to twitter api and results would be in JSON format that can be easily read by your program.
A good starting point would be reading the official documentation at https://dev.twitter.com/ itself.
Throw you can use and show some functionality in your website
is there any service from where we can download tweets?
UPDATE!!!
Googling for sometime gave me this result
a.) http://snap.stanford.edu/data/twitter7.html
b.) http://140kit.com/datasets
Yes, there is. It's called the Twitter API.
As we have access to limited tweets by Twitter-API, we should make use of third-party resellers like Topsy for just the past data, GNIP just for streaming data, or DataSift for both streaming data as well as past data.
You might also want to check the following sites:
http://www.infochimps.com/collections/twitter-census
http://www.tweetarchivist.com/
Twitter API allows provides partial results, it gives you the last 100 or even 500 tweets fo every search. If you need to keep tweets long term, twitter API shows its limits.
I had same need as you apparently hae and I developed a tool that queries twitter API periodically and stores search results on a Wordpress database.
I called the tools twittcorder and you can find a live demo on twittcorder.com
I hope this helps.
These other data sources are probably shared against the Twitter TOS. I wouldn't want to invest my time and effort building something on datasets that are non-repeatable. The Twitter Streaming API allows collection of a sample of Tweets.
There's also Gnip: http://gnip.com/.
Sysomos is there for complete data analysis including twitter, faecbook and various boards and forums
I'm trying to make use of various APIs including twitter, youtube, etc because we want to embed recent entries (tweets, videos) on our website.
However, since I'm just retrieving my own data, I'm wondering how I can do this simpler than the multi-step process required by OAuth.
Twitter provides me with my own access token I can use directly, so that kinda works, but I can't find any such token in the YouTube documentation.
So how am I supposed to make use of the api if I just want to get a simple list of stuff? how exaclty am I supposed to authenticate my own website to use my own account?
I think i might have things all wrong and if so please point me in the right direction. I tried using rss feeds but they don't give me as much control over what I retrieve as using the API directly...
any insight or suggestions are appreciated!
see my comment above. summary: it depends on the requirements of the individual api