I have this project that's based on a Twitter Bot. NOT an abusive or spammer bot. Just a robot that will reply with information about my system when it's mentioned.
The thing is: I expect to have about 3000 mentions/hour. Which means I have to send about 3000 replies/hour.
I know Twitter API calls are limited to 350/h when the app is authenticated and 150/h when it's not. But I've also read about it being 1000/h somewhere in the Twitter Discussion boards.
Anyway, how can I deal with having to send so many replies/hour?
I thought about creating multiple Twitter accounts and delegating some of the replies to them in order to being able to reply to all of them, but I do realize it's not a pretty solution.
Does anyone have a better idea?
Your best bet is to use Streaming.
Connecting
API Params (JSON)
Basically, you open up a connection with their server with your credentials and instead of reading the return stream and closing, you just keep reading. When you want to make a request, you just send another header or something.
Related
I am making an api backend that makes use of another api,for example Twitter. (Please note the actual api isn't twitter but I am using them as an example).
Let's say that Twitter has a limit on the number of calls that can be made to their api and above this limit, it starts to charge my credit card. This is why it is very important to me that no one misuses my api.
I want to prevent people from looking at my frontend code and seeing which endpoint it hits, because if a malicious person were to do this, I would very quickly go over the limit and have to pay $$$.
My frontend code uses a get call to mybackend.com/twitter/api
Is it enough to simply add an Access-Control-Allow-Origin header to my backend?
headers['Access-Control-Allow-Origin'] = 'myfrontend.com'
The reason I am asking this is because I noticed that typing mybackend.com/twitter/api directly into the browser worked, which is not what I would expect if I had access-control-allow-origin set to a specific website.
Am I doing something wrong? How do I prevent someone from simply writing a script to hit my backend since it is clear that just typing it into the url of my browser works, despite me having an access-control-allow-origin header.
There are two possible solutions for your problem. You can try to implement a request signature for your API, to know exactly the source of it on your backend. You can take a look on how this works here.
The second option, and for me, a one witch fits your problem better, is to set up a Denial of service approach on your server Load Balancer to prevent multiple requests from a same origin, and so, don't let those kind of malicious requests hit your backend.
I'm using pusher gem to manipulate my front-end from an external API. It works fine, no problem with that.
But the thing I wonder is if there is a possibility to use push notifications at the back-end of my application? I spent a serious amount of time investigating this but couldn't find something useful.
Let me summarize:
I have an application and another API application which is tightly interacting with other. Sometimes I want to use my API to send notification to my main application and I want to be able to manipulate data at the back-end of my main application regarding the data received from API side. These are things like 'an action was completed/started/succeed' etc...
I understand that 'pusher' receives push notifications by JavaScript at the front-end. But I believe that there must be a way to use those notifications at the back-end as well.
If there is another way (maybe Faye? Websocket) to do that I'd love to learn what it is. Any clue would be appreciated.
Is it something doable?
Thank you
Pusher is a backend system too (to "push" updates to channels)
Endpoints
I think you may be interested in endpoints
From what I can gather, it seems you're looking to trigger the transfer of data to an endpoint once an action occurs in your API? For example:
User signs up on "API" app
API app sends "notification" to main app
Main app increases user count by 1
The way I can see this working is by either using ajax, or sending a curl request to your main app's endpoint (set in routes), triggering the action:
#main_app/config/routes.rb
post "endpoint", to: "application#endpoint"
#main_app/controllers/application_controller.rb
def endpoint
#count = Option.increment!(:user_count)
end
This will allow you to manipulate your data in the backend of your "main" app
API
The tricky, non-conventional part comes when you want to send the data from your API app to your Main app (this is where you got the "pusher" idea from)
I would personally look at sending a standard HTTP request to the Main app endpoint, probably with Curl (if from the backend):
Curl on Ruby on Rails
Rails curl syntax
You may want to install curb (CUrl RuBy) here: https://github.com/taf2/curb
I could write some code if you wanted?
I had asked the same question to the Pusher's support team and I got the exact answer I was looking for.
You can install a client library on your server
(http://pusher.com/docs/client_libraries) if there is one for your
server. You can then subscribe to a client channel this way.
In my case, I use Ruby gem which can be reached from https://github.com/pusher/pusher-ruby-client .
I am new in Twitter API. I have a requirement like I have to pull the information of an authenticated twitter user to know the number of followers. I can do this using REST API but I have a problem here. So I have taken the way to use Twitter User Streams.
I am not able to integrate this functionality in the site. I am using twitter console( https://dev.twitter.com/console ). But there I am not able to get any kind of result, it actually hangs/shows gateway timeout. And that's for I am not able to configure that how to write the code for this or how to use this in site. I am not passing any kind of parameters.
So, any body can please help in this would be very grateful( a clean sample code for this or any link where to get the good documentation ). I have already checked with twitter dev documentations.
Thanks in advance :)
The question is simply the title.
Please don't tell me to go for a Twitter Data reseller like Gnip or Datasift as they're so expensive.
Thank you.
Run more than one twitter stream, there is no other way around to solve this issue. For running multiple streamer basically you need a dedicated twitter user, ip address and registered application for each streamer.
is it possible to submit forms with yahoo pipes?
i basically need to log in somewhere, and get some stuff from the members area of a website into a feed.
Although this is not exactly programming related... I guess it is close enough.
No, logging into somewhere is impossible with Yahoo Pipes. Sending the username/password isn't even the only problem here.
The real problem is that most, if not all, web sites that require a log-in depend on a session cookie or something similar. Yahoo pipes can do a GET request, and that's about it. Even if it was possible to send your user name/password in the URL, you would not be able to use the session cookie, so subsequent requests would fail.
So... If you have access to a hosted web site somewhere: Write a small proxy script (in PHP or whatever is available) that does the login and fetches the data. Let Yahoo pipes read from your proxy page. But if you are that far, you can just as well produce RSS format right away. ;-)
I did a pipe that can log in and extract info. is working ok on a simple web form using POST.