How can I use read only access to twitter API using OAuth authentication to crawl the twitter data like user timeline, followers, following-list etc. I used to use BasicAuth but now its not supported.
I don't know what platform you're working with, but the "how to connect with Oauth" problem has been solved multiple times on just about every platform. Rather than writing this code yourself (and giving yourself a thousand headaches), you may be better served finding one of the many Twitter libraries out there that already support OAuth.
Related
I'm working on a fairly basic Alexa skill that, in essence, searches through a specific Twitter feed looking for a hashtag, parses that tweet, and reads it back. The simplest way to do this seems to be using the Twitter API, since scraping appears to be against the TOS.
... crawling the Services is permissible if done in accordance with the provisions of the robots.txt file, however, scraping the Services without the prior consent of Twitter is expressly prohibited.
I've been having some trouble understanding how account linking works, as I've never dealt with OAuth before. I've been trying to follow the one tutorial around, but neither the text or video version were clear me.
Why the need for an external webapp?
...we need an OAuth implementation of our own to make the integration work correctly
What's wrong with the one provided by Twitter? Why can't any issues be fixed within the Lambda method, since the account integration isn't being touched otherwise AFAIK? Isn't having the tokens passed around via the URL a bad idea too? Their example code seems to require that the Consumer Secret be hard coded too.
Enter: “https://alexa-twitter-airport-info.herokuapp.com/oauth/request_token?vendor_id=XXXXXX&consumer_key=YYYYYY&consumer_secret=ZZZZZZ”.
At the very least, their webapp seems to be down for the time being, and it'd be nice to have an option that doesn't require paying money to host another copy.
I've seen this post discussing a Node.js OAuth implementation, but the necessity for such an implementation still escapes me.
Recently, YouTube decided to make video tags unavailable publicly. So to get the tags for a given video, I need to make an authenticated request to the API as the owner of the video. This is not a problem in my case as I'm fetching my own videos.
However, I'm confused about the authentication flow since YouTube strongly recommends to use OAuth2. Since I'm always going to authenticate as the same user (the owner of the video, aka myself), I definitely don't need to have any browser page for the actual user of the app to do anything. I see how I could have done it using ClientLogin (hardcoding login and password into the app) but I'm not sure how to approach this using OAuth2.
One last detail - that is not necessarily relevant since a high-level answer would be enough - is that I'm developing on iOS. Also I looked at this https://developers.google.com/accounts/docs/OAuth2 and particularly the web server case which seems closest to mine but was not able to get a clear idea from it.
Thanks in advance for your help and don't hesitate if you need me to be more specific.
There is no OAuth flow that supports your use case.
In general, you should not be distributing your YouTube login as part of your application. Even if this were available via ClientLogin, after a certain number of logins, you would likely be presented with a challenge because the authentication servers would detect a strange usage pattern.
OAuth is not for distribution a single user's login to a large N, where N is the number of users of your application. OAuth is meant for your application to act on behalf of an end user, and because tags are no longer exposed to end users through the UI, it does not make sense to expose them to users via the API either. More details can be found here:
http://apiblog.youtube.com/2012/08/video-tags-just-for-uploaders.html
How many videos do you have? What is the purpose for needing the tag metadata? From a pragmatic perspective, here are a few alternative implementations that would be easier and would not require users to log in as you:
Store a single file mapping video IDs to tags on a server somewhere and fetch this periodically. Google App Engine is a good place to do this.
Put the tag data in the description in a predictable format (you host the videos), and generate the metadata from this.
My question is not about FourSquare API and its functions, but about more simpler details that are not well explained on Foursquare API explanations. Thank you very much in advance if someone wants to help me with this doubts:
Foursquare API is a framework you can use to build applications for mobile devices, above of IOS and Android, so i can imagine that they have API for IOS (Objective-C) and Android (Java), right?
From API Doc: "Be sure to note that although API requests are against api.foursquare.com, OAuth token and authorization requests are against foursquare.com.". Does it mean that if i want to use FourSquare app, the users has to have an account on FourSquare?
From API Doc: "For example, if you write an iPhone application, every user who logs in with their foursquare account will be able to make up to 500 users/* requests and up to 500 venues/* requests, etc." I dont understand this sentence. Does it mean that for example, if you use an API method request like "checkins.add()", this method create two methods? one against api.foursquare.com to monitor the API limit requests, and another to your Web Application Server?
So as a question related to the third one, where do you have to store your database? is it stored on Foursquare cloud database because you are loggin there, or you have to create your own Web Service application with its own SQL database?
From API Doc: "All requests are simple GET or POST requests that return JSON or JSONP respones", so i can imagine that the Web Application Service should understand JSON. Well, my main question is, can i use Ruby on Rails to build the Web Application Service and Web Page frontend? I am seeing that there are some wrapps for RoR designed from third companies, but are not official and doesnt cover all the 2.0 API, just the ones they needed for their services.
If i want to create an app using FourSquare API, what do you advice me to use as a programming language/framework for the Web Service Application? the WSA that has to process the JSON requests and later store them on the database, interaction with users on the WebPage, etc.
i am so sorry if my questions are so simple, but i dont have any other place of this level of expertise.
thank you very very much in advance.
The API is REST/JSON based, which means that any language that can do an HTTP request and parse a string can be used. There are Java and iOS libraries available. But you could use just about anything - curl with bash would be a bit extreme but if that floats your boat...
For some of the APIs (search a venue, for example) you do not necessarily need a FourSquar OAuth user token. For others (like checkin) a FourSquare token is required. For any API calls that require a userid, your users will have to be FourSquare users and "trust" your application with their FourSquare data.
Only requests to FourSquare is counted. So if you do a single call to checkins.add() it counts as one call for the user that is doing the checkin. I wouldn't worry about the limits. As long as you're usage of the API is sensible they will not be a problem. And if they do become a problem and you're doing something extraordinarily cool, the folks at FourSquare might be sympathetic.
You have to create your own web server with your own database to store some information. The OAuth token is one. You probably want to cache venue information here for short periods as well.
Yes, your webapp will need to be able to understand JSON. Ruby has excellent JSON support - look for the json gem.
It is really difficult to suggest a language or framework without knowing what it is that you're trying to do. I wouldn't choose a framework based on the fact that you want to use FourSquare (anything will do) but rather on your experience and the unique features of your application. You mentioned RoR before - that would definitely work.
I'm using https://github.com/jnunemaker/twitter to tweet to a users twitter when they post on their blog running on ROR. .e.g
Tweet : "I just posted a blog - 'I love ruby on rails' http://link-to-blog.com"
My question is, as I'm making many sites for different people do I have to create a new twitter developer application, with individual consumer keys & secrets, for each blog or is there a way to use the same twitter application?
Thanks,
Alex
You technically can use the same application in a variety of websites. Just use the keys/tokens twitter gives you in all your sites.
Nonetheless, this is a bad practice, since twitter will not be accounting your accesses to the API from the pages that are not the one you specify in the Callback URL. Furthermore, your users will return to that (and only to that) page that you specified in the callback URL, which can be very misleading for those that are in other site.
And finally the most important reasons are the following two:
You'll get to the request limit quicker than if you had several applications
You'll get to the user limit quicker than if you had several applications
The limits that twitter manages are not very big so I can tell you that the twitter functionalities won't work if you get a good peak of visits (happened to me twice). Or they may not work if you're site receives a lot of visits at a certain time. No matter if your cache your API or not, you'll end up filling the limit.
Here is the twitter documentation about this:
Caching. We recommend that you cache API responses in your application or on your site if you expect high-volume usage. For example, don't try to call the Twitter API on every page load of your hugely popular website. Instead, call our API once a minute and save the response to your local server, displaying your cached version on your site. Refer to the Terms of Service for specific information about caching limitations.
Rate limiting by active user. If your site keeps track of many Twitter users (for example, fetching their current status or statistics about their Twitter usage), please consider only requesting data for users who have recently signed in to your site.
Scale your use of the API with the number of users you have. When using OAuth to authenticate requests with the API, the rate limit applied is specific to that user_token. This means, every user who authorizes your application to act on their behalf, has their own bucket of API requests for you to use.
Request only what you need, and only when you need it. For example, polling the REST API looking for new data is inefficient for both your application, and the Twitter API. Instead consider using one of the Streaming APIs as a signal of when to make REST API requests.
If you have any question, don't hesitate to comment below. I had terrible experiences with this when my site got mentioned by a few important twitter accounts
I want to put the Twitter service on my server and customize it for my purpose. I have no idea how it works.
My goal is to communicate to your own Twitter server rather than the original twitter server and serve my purpose.
You should check out: StatusNet. It is an open source micro blogging platform. From their site, you can download the source and deploy it on your own server. Once you have it installed you can customize it to your liking.
Twitter isn't an Open Source project - they don't provide their server code.
From my experience at another company deploying very widely distributed systems, the chances are there's a bucket-load of infrastructure you'd need to get running first - complete overkill for a single-server solution, but vital for a global service with many millions of users. In other words, even if Twitter did provide their code, it probably wouldn't be an appropriate solution for your situation.
The actual Twitter (twitter.com) service is proprietary, you can't run it yourself.
There are plenty of open source twitter clones out there. The more general name is "microblogging". Pinax for example has basic microblogging. Try searching google for 'open source microblogging' for other projects.
I don't believe the Twitter platform is freely available to the general public. If you want to make your own "Twitter server", you're going to have to clone the service yourself.
You can't run Twitter on your own server, but you can write your own application that talks to Twitter through Twitter's API.
It all depends on what you mean by "customizing" Twitter. There are many applications like Twitpic and TweetDeck that are built "on top of" Twitter. They add their own functionality while leaving Twitter to do the "heavy lifting".
For example, I have written a personal project for moderating a stream of tweets. This application runs on my local server, but it gets its data by querying Twitter's API.
There are two main advantages to extending rather than rebuilding Twitter:
It takes a lot less effort because you can reuse all the basic functions of Twitter
You can take advantage of Twitter's huge user base. Even if you succeeded in cloning Twitter, it would be far less interesting than the original because Twitter works by strength of numbers.
You could use Wordpress and get the twitter developer add in then get a api code from them and there users can use your site and vice versa also apps for twitter will work for your site.
Wow. That's a highly ambitious request that you have there. Twitter isn't like Wordpress, there's no .org version that can be downloaded and run locally. Twitter is a highly scalable service that is designed to run on large scale servers.
Sorry to be the bearer of bad news to you on this.