I have a website that pays users for showing advertising on shortened links. I recently have someone who is trying to boost his earnings by sending fake traffic to the site. It is not only smashing my site (ie. slowing it down) but is also skewing my Google Analytics data.
Is the only solution here really to deploy a WAF, or roll my own Rack based request inspection code? I not sure how this would work or best practices in a Heroku/Ruby on Rails environment.
Any one have experience on this or have any ideas?
If the requests are being sent at a much higher rate than regular visitors do, the rack-attack gem might be very useful to you.
It allows throttling requests, so if a user visits some page at a higher frequency than allowed for a while, you can (temporarily) block any further requests.
From the gem's README:
Rack middleware for blocking & throttling abusive requests
Rack::Attack is a rack middleware to protect your web app from bad clients. It allows whitelisting, blacklisting, throttling, and tracking based on arbitrary properties of the request.
Do define the rules with a safe margin, so your application won't be blocking regular users who aren't spamming your site.
Related
We have a Rails app with a view that gets populated with data from a third-party API. Currently, this view uses a swf streamer to open a socket to the endpoints.
Recently, the API's provider has asked us to switch to long polling ajax calls, and to pipe requests through a proxy in our server.
We're considering using node-http-proxy, to take advantage of node's speed and concurrency handling in case we get high traffic. We're new to Node.js, though.
The other option we're looking at is using the Rails app itself to forward these requests, the advantage being that we could use the existing session handling.
We'd prefer to use node-http-proxy, as it seems the most elegant solution (and an opportunity to play with Node.js, of course ;), but haven't figured out how to integrate it with our app's sessions (activerecord session store on postgres).
Is there a way to do it? Are there any other auth/security/session-checking strategies using node-http-proxy in parallel with a Rails app?
Oliver
I am using Rails 3.1.1 and I would like to redirect users, for example, from the U.S.A. to a proper subdomain us.site.com (this one is hosted on the same server as site.com). I know that I can localize a user by his/her IP address but how can I do that so to redirect he/she to the proper subdomain? There is a technique/gem to geo-locate user IPs and then handle redirection?
P.S.: Maybe, for performance reasons, I should use middlewares...
https://rubygems.org/gems/rack-geoipcity is a rack middleware gem I've published which you could use, or just use the GeoIP gem in your controllers.
With the rack-geoipcity, you would query the X-headers it adds in and make a decision based on that. Something like:
if headers['X_GEOIP_COUNTRY_CODE'] == "IN"
redirect "/india"
end
though I don't currently use Rails, so it might be slightly different.
There are plenty to choose from if you don't fancy using the MaxMind db.
One approach that I've used happily in the past is to perform the geolocation lookup via DNS before users connect to the service; this way, they automatically connect to the server nearest them, you get cheap and easy load balancing and ability to remove servers from active use as you need, and individual sites can go down without influencing other sites.
OFTC uses a self-written oftcdns tool to provide users with nearest servers. During the time I was an administrator on the OFTC network, this tool was a drastic improvement over running a simpler Bind-based DNS server that did not provide geo-location features and complicated bringing servers in and out of the rotation.
Wikipedia uses PowerDNS with a geobackend to provide their geo-ip services. PowerDNS is definitely well-tested, high-demand-capable tool.
I'm using https://github.com/jnunemaker/twitter to tweet to a users twitter when they post on their blog running on ROR. .e.g
Tweet : "I just posted a blog - 'I love ruby on rails' http://link-to-blog.com"
My question is, as I'm making many sites for different people do I have to create a new twitter developer application, with individual consumer keys & secrets, for each blog or is there a way to use the same twitter application?
Thanks,
Alex
You technically can use the same application in a variety of websites. Just use the keys/tokens twitter gives you in all your sites.
Nonetheless, this is a bad practice, since twitter will not be accounting your accesses to the API from the pages that are not the one you specify in the Callback URL. Furthermore, your users will return to that (and only to that) page that you specified in the callback URL, which can be very misleading for those that are in other site.
And finally the most important reasons are the following two:
You'll get to the request limit quicker than if you had several applications
You'll get to the user limit quicker than if you had several applications
The limits that twitter manages are not very big so I can tell you that the twitter functionalities won't work if you get a good peak of visits (happened to me twice). Or they may not work if you're site receives a lot of visits at a certain time. No matter if your cache your API or not, you'll end up filling the limit.
Here is the twitter documentation about this:
Caching. We recommend that you cache API responses in your application or on your site if you expect high-volume usage. For example, don't try to call the Twitter API on every page load of your hugely popular website. Instead, call our API once a minute and save the response to your local server, displaying your cached version on your site. Refer to the Terms of Service for specific information about caching limitations.
Rate limiting by active user. If your site keeps track of many Twitter users (for example, fetching their current status or statistics about their Twitter usage), please consider only requesting data for users who have recently signed in to your site.
Scale your use of the API with the number of users you have. When using OAuth to authenticate requests with the API, the rate limit applied is specific to that user_token. This means, every user who authorizes your application to act on their behalf, has their own bucket of API requests for you to use.
Request only what you need, and only when you need it. For example, polling the REST API looking for new data is inefficient for both your application, and the Twitter API. Instead consider using one of the Streaming APIs as a signal of when to make REST API requests.
If you have any question, don't hesitate to comment below. I had terrible experiences with this when my site got mentioned by a few important twitter accounts
I'm trying to create a ruby on rails ecommerce application, where potential customers will be able to place an order and the store owner will be able to receive the order in real-time.
The finalized order will be recorded into the database (at the moment SQLite), and the storeowner will have a browser window open, where the new orders will appear just after the order is finalized.
(Application info: I'm using the HOBO rails framework, and planning to host the app in Heroku)
I'm now considering the best technology to implement this, as the application is expected to have a lot of users sending in a lot of orders:
1) Each browser window refreshes the page every X minutes, polling the server continuously for new records (new orders). Of course, this puts a heavy load on the server.
2) As above, but poll the server with some kind of AJAX framework.
3) Use some kind of server push technology, like 'comet' asynchronous messaging. Found Juggernaut, only problem is that it is using Flash and custom ports, and this could be a problem as my app should be accessible behind corporate firewalls and NAT.
4) I'm also checking node.js framework, seems to be efficient for this kind of asynchronous messaging, though it is not supported in Heroku.
Which is the most efficient way to implement this kind of functionality? Is there perhaps another method that I have not thought of?
Thank you for your time and help!
Node.js would probably be a nice fit - it's fast, loves realtime and has great comet support. Only downside is that you are introducing another technology into your solution. It's pretty fun to program in tho and a lot of the libraries have been inspired by rails and sinatra.
I know heroku has been running a node.js beta for a while and people were using it as part of the recent nodeknockout competition. See this blog post. If that's not an option, you could definitely host it elsewhere. If you host it at heroku, you might be able to proxy requests. Otherwise, you could happily run it off a sub domain so you can share cookies.
Also checkout socket.io. It does a great job of choosing the best way to do comet based on the browser's capabilities.
To share data between node and rails, you could share cookies and then store the session data in your database where both applications can get to it. A more involved architecture might involve using Redis to publish messages between them. Or you might be able to get away with passing everything you need in the http requests.
In HTTP, requests can only come from the client. Thus the best options are what you already mentioned (polling and HTTP streaming).
Polling is the easier to implement option; it will use quite a bit of bandwidth though. That's why you should keep the requests and responses as small as possible, so you should definitely use XHR (Ajax) for this.
Your other option is HTTP streaming (Comet); it will require more work on the set up, but you might find it worth the effort. You can give Realtime on Rails a shot. For more information and tips on how to reduce bandwidth usage, see:
http://ajaxpatterns.org/Periodic_Refresh
http://ajaxpatterns.org/HTTP_Streaming
Actually, if you have your storeowner run Chrome (other browsers will follow soon), you can use WebSockets (just for the storeowner's notification though), which allows you to have a constant connection open, and you can send data to the browser without the browser requesting anything.
There are a few websocket libraries for node.js, but i believe you can do it easily yourself using just a regular tcp connection.
I'm running a Rails app on the Heroku Stack (complete with Memcached, DJ Asynchronous workers, MongoDB persistent storage).
Right now we use Twitter Oauth as the only authentication option on our site. (We plan to branch out to FB connect, OpenID, and/or Email/password eventually).
Ruby/Rails apps, as you probably know, don't support concurrency out of the box. On Heroku, you can spin up additional app instances (dynos), which increase your concurrency (concurrency capability = number of dynos), but each one costs $36/month.
In general, this hasn't been a problem, because the average request on the site takes <100 ms.
EXCEPT for Twitter OAuth. The OAuth-related requests to Twitter take, on average, around 3,500 ms.
So basically, when anyone logs in an entire app instance gets held up for 3-4 seconds.
Is there any decent way to mitigate this? Would it be weird to put these actions in asynchronous DJ workers? It could make logging in a little slower, but at least if a bunch of people are logging in at once and/or Twitter is being really slow, these processes do not affect the rest of the app / other web requests?
Any other ideas?
I would say that this advice is now superseded. I'd suggest you use OmniAuth instead, possibly with Devise if you need normal auth as well.
OmniAuth is a Rack app as suggested and you basically get all the "big" OAuth providers working in fell swoop.
There are 2 OmniAuth specific RailsCasts that walk you through exactly what's needed: OmniAuth Part 1 and OmniAuth Part 2
You could push the oAuth into a rack middleware app - then at least only a rack app gets spooled up instead of the whole rails stack. That should make it a little quicker (though it will still take up an instance).
That being said, there is no reason why you can't put authentication into it's own app, especially if it's all in the same domain (so any authentication cookies are still local to the domain). Though you'll have to be really careful of security issues - eg man-in-the-middle attacks etc. Better if somebody's already done the work/bugfixing for you :)
You can even have an authentication app on an independent app domain if you use rubyCAS: http://rubyglasses.blogspot.com/2009/12/rails-single-sign-on-with-rubycas.html