we are running rails app in production (single master node) with nginx as web and puma as rack server and wanted to calculate no of request our server can handle. I know there are tools available like ApacheBench which works like
ab -k -c 350 -n 20000 example.com
It takes few parameters like in above command 350 simultaneous connections until 20 thousand requests. This approach can give req per seconds count for a single URL. But, I am interested in determining request per seconds for dynamic system which serves dynamic content.
Is there any built-in tool which can give me request per second count.
Way Around (Manual Calculations)
I have installed an Analytics tool rorvswild it is very similar to NewRelic. It gives me response time of every route I have in ruby on rails application. It also gave average response time of all the routes, which is 250ms If average response time of system is T can I calculate no of request system can handle will be 1000/T?
Also I am running puma behind NGINX, which is multithreaded and running 5 threads so eventually
no of requests per second = thread_count * (1000/T)
In my case thread_count = 5
Thank You so much for reading, you suggestions will be very helpful.
Related
I recently changed from passenger to puma because it was constantly giving me "request limit exceeded error" and I read online that passenger free version doesn't support multithreading. My backend application is hosted on two AWS c5xlarge instances and has and elastic load balancer on top. Can someone help me with the number of workers and threads I should set in puma config and a maximum number of concurrent requests I can serve with these settings?
There is no clear answer to your question. It depends on a lot of parameters.
You should create a benchmark script which sends a lot of requests from multiple process and/or threads, and if your server can handle a heavy load even from multiple instances, and see how many requests are served in a second.
After you have this benchmark test try to change the number of threads and workers, in order to increase the number of requests served using this benchmark.
I would start with one nginx worker and the number of threads as the number of cpu cores and increase or decrease according to the benchmark.
I'm trying to scale up an app server to process over 20,000 requests per minute.
When I stress-test the requests, most requests are easily handling 20,000 RPM or more.
But, requests that need to make an external HTTP request (eg, Facebook Login) bring the server down to a crawl (3,000 RPM).
I conceptually understand the limitations of my current environment -- 3 load-balanced servers with 4 unicorn workers per server can only handle 12 requests at a time, even if all of them are waiting on HTTP requests.
What are my options for scaling this better? I'd like to handle many more connections at once.
Possible solutions as I understand it:
Brute force: use more unicorn workers (ie, more RAM) and more servers.
Push all the blocking operations into background/worker processes to free up the web processes. Clients will need to poll periodically to find when their request has completed.
Move to Puma instead of Unicorn (and probably to Rubinius from MRI), so that I can use threads instead of processes -- which may(??) improve memory usage per connection, and therefore allow the number of workers to be increased.
Fundamentally, what I'm looking for is: Is there a better way to increase the number of blocked/queued requests a single worker can handle so that I can increase the number of connections per server?
For example, I've heard discussion of using Thin with EventMachine. Does this open up the possibility of a Rails worker that can put down the web request it's currently working on (because that one is waiting on an external server) and then picks up another request while it's waiting? If so, is this a worthwhile avenue to pursue for performance compared with Unicorn and Puma? (Does it strongly depend on the runtime activities of the app?)
Unicorn is a single-threaded, multi-process synchronous app server. It's not a good match for this kind of processing.
It sounds like your application is I/O bound. This argues for an event-oriented daemon to process your requests.
I'd recommend trying EventMachine and the em-http-request and em-http-server.
This will allow you to service both incoming requests to the http server and outgoing HTTP service calls asynchronously.
I've found the following at Docs: Scaling Puppet:
Are you using the default webserver?
WEBrick, the default web server used to enable Puppet’s web services connectivity, is essentially a reference implementation, and becomes unreliable beyond about ten managed nodes. In any sort of production environment serving many nodes, you should switch to a more efficient web server implementation such as Passenger or Mongrel.
Where does the the number 10 come from in "ten managed nodes"?
I have a little over 20 nodes and I might soon have little over 30. Should I change to Passenger or not?
You should change to Passenger when you start having problems with WEBrick (or a little before). When that happens for you will depend on your workload.
The biggest problem with WEBrick is that it's single-threaded and blocking; once it's started working on a request, it cannot handle any other requests until it's done with the first one. Thus, what will make the difference to you is how much of the time Puppet spends processing requests.
Each time a client asks for its catalog, that's a request. Each separate file retrieved via puppet:/// URLs is also a request. If you're using Puppet lightly, each catalog won't take too long to generate, you won't be distributing many files on any given Puppet run, and each client won't be taking more than four to six seconds of server time every hour. If each client takes four seconds of server time per hour, 10 clients have a 5% chance of collisions0--of at least one client having to wait while another's request is processed. For 20 or 30 clients, those chances are 19% and 39%, respectively. As long as each request is short, you might be able to live with some contention, but the odds of collisions increase pretty quickly, so if you've got more than, say, 50 hosts (75% collision chance) you really ought to by using Passenger unless you're doing active performance measuring that shows that you're doing okay.
If, however, you're working your Puppet master harder--taking longer to generate catalogs, serving lots of files, serving large files, or whatever--you need to switch to Passenger sooner. I inherited a set of about thirty hosts with a WEBrick Puppet master where things were doing okay, but when I started deploying new systems, all of the Puppet traffic caused by a fresh deployment (including a couple of gigabyte files1) was preventing other hosts from getting their updates, so that's when I was forced to switch to Passenger.
In short, you'll probably be okay with 30 nodes if you're not doing anything too intense with Puppet, but at that point you need to be monitoring the performance of at least your Puppet master and preferably your clients' update status, too, so you'll know when you start running beyond the capabilities of WEBrick.
0 This is a standard birthday paradox calculation; if n is the number of clients and s is the average number of seconds of server time each client uses per hour, then the chance of having at least one collision during an hour is given by 1-(s/3600)!/((s/3600)^n*((s/3600)-n)!).
1 Puppet isn't really a good avenue for distributing files of this size in any case. I eventually switched to putting them on an NFS share that all of the hosts had access to.
For 20-30 nodes, there shouldn't be any problem. Note that passenger provides some additional features. It may be faster serving the nodes, but I am not sure how much improvement you will get if you have only 30 nodes.
You should change to passenger if you are using more than hundred nodes. I started seeing problems when the number of nodes requesting service from the puppet-master reached about 200. In my case, with the default web-server, about 5% of the nodes (random) couldn't receive the catalog during hourly run.
I'm new to Heroku, and would like to have some idea of how I can go about guesstimating the number of dynos that might be needed for a RoR app (need to give some number to customer).
The app is already running on the free 1 dyno. Here's some basic info about the app:
App is mainly database reads, with very little writes
Possibly heavy DB load, with queries involving distance calculations (from lat-long, using gmaps4rails)
From some basic testing with WAPT (eval version), it looks like a typical search request takes a min. ~1.3s, avg. ~2s, max. 4-5s
Again from WAPT testing, up to 20 concurrent users and observing the Heroku logs, I don't seem to be seeing any requests being queued
Other requests are largely static assets
How would I get some rough idea of the number of dynos needed, to handle X concurrent users, or how many concurrent users the single dyno can likely handle?
Update: Heroku changed their dyno pricing prolicy https://www.heroku.com/pricing, so the these information might not correct anymore.
According to this article http://neilmiddleton-eu.herokuapp.com/getting-more-from-your-heroku-dynos/, if you use Unicorn, 1 dyno can handle 1 million request per day (100ms per request). So if you host all media in S3, 1 page view need 3 requests (1 html, 1 pipe-lined css, 1 pipe-lined javascript), 1 dyno can handle roughly 300.000 page view a day, or 80 page view per seconds with Unicorn.
Let's say 1 user will view 1 page in 5 seconds, and your application can manage to respond in 300ms, technically, you will have roughly 400 concurrent users with 1 dyno.
But actually our application (quite heavy), 1 dyno can only accept 1/10 of those, around 50 concurrent users.
Hope this help you!
If using in the controller:
def index
#time_start_in_controller
...
end
and at the end of view, use
<%= "took #(Time.now - #time_start_in_controller} seconds" %>
but isn't the time at the view not the true ending of rendering, because it needs to mix with the layout and so forth. What is a more accurate way (just as accurate as possible) to print out the page generation time right on the webpage?
(update: also, the console showing the log as taking 61ms, but the page definitely took 2 to 3 seconds to load, and the network I am using is super fast, at home or at work, at 18mbps or higher with a ping of maybe 30ms)
update: it is a bit strange that if I use the http performance test ab
ab -n 10 http://www.my-web-site.com:8080
it takes 3 seconds total for 10 requests. But if I use Firefox or Chrome to load the page, each page load is about 3 seconds. This is tunneling to my work's computer back to my notebook running Rails 3, but shouldn't make a difference because I run Bash locally for the above statement and use Firefox locally too.
in a typical production environment, static content (images, css, js) are handled by the web server (eg. apache, nginx etc) not you rails server. so you should check their logs as well. if you are serving static content from your rails server that could be your problem right there.
If your browser time is slow but the time taken in rails (According to the logs) is fast, that can mean many things including but not quite limited to:
network speed is slow
your dns server is slow and browser can't resolve your dns quickly (this happens with for instance if you use godaddy for your dns server they throttle dns lookups)
the requests concurrency exceeds how many threads you have in rails
one way to debug these types of performance issues is to put something in front of the rails server (for example Haproxy) and turn the logging to full. As they will show how many waiting requests there are and how long the actual request/response transferring took along with how long it took your rails thread to process.