I have a web application on RoR 2.1 and backend MySQL up and running with around 8k users and now i want to do a Load Test on my web app and server to figure out the load on the server and the average and peak number of concurrent users.
What are the ways of implementing this load test to analyse the load on the server and performance of the web application with a way to figure out average and peak number of concurrent users?
I'm using ab (apache benchmarks http://httpd.apache.org/docs/2.0/programs/ab.html) for load tests. Example of testing on google.com:
ab -n 10000 -c 100 http://google.com/
It allows me to investigate how much requests per second my setup(application) can do as well as concurrency level.
The ab tool is a part of the Apache httpd package in CentOS and Red Hat distributions. So it is probably already installed there. For Ubuntu/Debian install apache2-utils package.
ab --help for full options list
The most important are :
-n requests Number of requests to perform
-c concurrency Number of multiple requests to make
Also i'm monitoring peaks of activity with munin(http://httpd.apache.org/docs/2.0/programs/ab.html) and plugins for nginx/passenger/unicorn/CPU/Memory depending on configuration, as well as plugin for MySQL which shows total amount of queries per second and many more data.
You can install munin using appropriate tutorial for your RH linux from that page http://munin-monitoring.org/wiki/LinuxInstallation.
Here also quite nice article about munin and mongrel monitoring:
http://onrails.org/2007/08/31/monitoring-rails-performance-with-munin-and-a-mongrel
You could pick up plugins for apache(and not only) monitoring from the http://exchange.munin-monitoring.org.
Good thing about that all that it doesnt require to change application. So you can just install it and use without any changes from your production setup.
Related
I've installed Neo4J 3.2.1 server on an Ubuntu 14.04 machine with 16G of RAM. I'm connecting to this Neo4J server from my Windows 10 laptop via browser (http://server_ip_address:7474).
I'm working with 3 different graphs with 280MB, 6MB and 13MB. No problem when I activate one of the two smaller ones but impossible to work (make cypher queries in the NEO4J-browser) with the larger because it blocks for long periods (over one minute each).
I've tried to fix this myself. Read https://neo4j.com/docs/operations-manual/current/performance/ and set the following values in neo4j.conf file: - dbms.memory.heap.initial_size=12g; - dbms.memory.heap.max_size=12g; - dbms.memory.pagecache.size=4g.
I also tried: - echo noop > /sys/block/sda/queue/scheduler; - mount -o remount,noatime /.
Since none of these attempts worked, I would appreciate some help!
I'm not sure but it seems that there's some kind of cyclic script running causing the delays.
I only have trouble using the http browser server. I wrote a simple java application to make remote queries without using the http browser, produced a executable jar with it and when I run that jar and make the queries the response times are normal.
I'm runnig Python Django on Apache2 with ModWSGI and I would like to run Ruby on Rails on the same server.
Is it possible do this?
I read that Passenger for Ruby on Rails can support Django too.
Any help is welcome.
Thanks.
Yes, I've done it. I would not suggest doing it for a server that is busy or prone to load spikes because of the way these modules work, they embed code into the server process itself (though mod_wsgi is better than mod_python for this).
When I do HEAD http://my.dev.server the response headers include this:
Server: Apache/2.2.8 (Ubuntu) DAV/2 Phusion_Passenger/2.2.15 PHP/5.2.4-2ubuntu5.10 with Suhosin-Patch mod_wsgi/1.3 Python/2.5.2
For example, if you have 40 apache child processes adding mod_wsgi may add a couple megs of RAM to each, so used RAM may go up by 100MB. Adding phusion passenger made cause each child to use another 5 MB of RAM increasing it by 200MB more.
However, if you have 10 apache child processes this is not a huge deal.
If you are on a server that gets these busy periods then one way to achieve better results is to use nginx instead of Apache (or fastcgi with apache). It does not embed any of the application code into the http process.
What do I need to get from a webhoster if I want to host my own Diaspora* seed? Do I just need some webspace with ruby on rails capabilities? I suppose it uses Ruby On Rails SOAP for communication between seeds? If so what is needed for this?
According to the README on GitHub, you need Ruby, MongoDB, OpenSSL, ImageMagick, Git, RubyGems, and Bundler... so any decent shared hosting that allows you to build things in your homedir and run custom long-running processes should do the trick.
(Decent because the non-decent ones may omit Git or ImageMagick and not give you the resources to install them yourself. Long-running processes for MongoDB and the Diaspora app server.)
WebFaction, for example, should meet those needs but Dreamhost may not unless the Diaspora app server can somehow be managed by FastCGI to work around their long-running process killer. (Dreamhost apparently supports MongoDB, so that would be exempt from the killer)
Seriously though, I'd recommend waiting. Diaspora was just released. It's still in development and, if anything, probably best to be experimented with on your home PC for at least a few more months.
In general you need a server which allows you to run background processes, as diaspora* uses Sidekiq to process background jobs.
Any VPS Linux server for example with a recommended minimum 1GB should do fine for a small user base.
See: https://wiki.diasporafoundation.org/FAQ_for_pod_maintainers#What_are_the_general_system_requirements.3F
It is the first time I have ever launched live a website (with Grails web framework under Amazon EC2 platform and Cloud Foundry) and I realized quickly that I am not ready for monitoring and maintening correctly my application in production mode (fortunately the website is accessible to a very limited number of users) .
The issues I have faced so far are:
Cannot change my views. I need to redeploy my application
I have no monitoring. I don't know who is connected, when do they sign in / sign out...
Redploying my application (upload WAR + deploy) takes at least 30 minutes.
I don't know how to restart my Tomcat server without a redeploy through Cloud Foundry !
...
So, my question is very simple:
What tools (including grails plugins) and methods can you recommend me for taking me out from my current blindness?
I am not sure how much this will help, however I use the JavaMelody Grails Plugin(http://www.grails.org/plugin/grails-melody) I use it to see if the site is being used before I pull down the service.
Hope that helps.
I tried Cloud Foundry in it's early days and found it a little rough. Sounds like that's still the case with 30+ minute deploys and the inability to restart your tomcat server. Half of your problems could be solved if you just created your own EC2 instance, installed tomcat and managed your own deployments. That'll let you bounce tomcat through shell access:
sudo /etc/init.d/tomcat6 restart
and redeploy your app
sudo /etc/init.d/tomcat6/stop
cp my.war /to/tomcat/dir
sudo /etc/init.d/tomcat6 start
or else you could do it through the tomcat admin console, but I find it to be flaky.
Regarding monitoring, there are a couple of ways you can do that. The easiest is to add CloudWatch monitoring to your system. That'll give you more insight into the performance of the application.
For more detailed monitoring regarding who's connecting to your app, I'd suggest looking at Google Analytics or Mint. If you need to get beyond that (with per user monitoring), you'll likely have to roll your own logging/tracking for what meets your needs. There are also other paid packages out there along the lines of Google Analytics and Mint that you can integrate with, but what fits your needs best, I can't say.
For actual monitoring of deployed system you can also use Hiperic HQ. It's a monitoring solution from Spring Source, who also are owners of Grails Framework.
It can manage, at your case:
tomcat server
database
linux
network
etc
btw redeploying app with changes is ok. it's a very bad practice to modify running app, on the production server.
I don't know if JMX and JConsole can help, but that might be a good way to see what the status of JMX-enabled POJOs is. Spring makes this easy to do.
Yes, you need to re-deploy your app when you change stuff in it, there's no way around that.
Deploy/re-deploy time has been cut significantly recently if using the grails plugin (btw, what version of the plugin do you use?) In some cases the upload time is as short as 15 seconds. Add 2-3 minutes for Amazon to spin up the instances and the deploy time is still pretty manageable. For re-deploy the instances don't have to be started, so it's even less than that.
To restart Tomcat login into your CloudFoundry account, click on the Deployment details, click on the instance that's running your Tomcat, and there will be a button "Restart service" that will do just that - restart Tomcat service.
You don't have to start your own EC2 instances in order to get a shell access. Copy the public DNS name of the instance from DeploymentDetails, and SSH into it using the private key you entered when registering for Cloudfoundry. Example:
ssh -i /path/to/gsg-keypair.pem root#your_instance_DNS_name
Okay, before you guys go nuts -- this is just a small site, temporary setup. Right now I'm having some internal folks remote into the server and use the site through webrick via the dev command: ruby script/server. Not exactly ideal.
I'm just starting Rails dev and I want to know a better way to handle hosting on a Windows Pro box. Again, just temporary so please be gentle :)
As far as I know, mod_rails isn't an alternative.
Mongrel plays very nice on windows, though, so you can set up a few mongrels and have IIS or Apache proxy to them. Or just use Mongrel directly. Before mod_rails, mongrel was the de facto way to deploy on any platform, so it's a very viable choice.
The one time I was forced to deploy on Windows, however, I installed Ubuntu via virtualbox (could also use VMWare or whatever, of course) and deployed on that. Works like a dream, and I got to work with a sensible OS. Phew. SSH and stuff. Can't live without it. Remote desktop isn't exactly my kind of thing.
Your best bet is to setup a Mongrel cluster. Mongrel is an application server which can serve a Rails application on HTTP. But a single Mongrel instance can only handle 1 request at the same time, so typically people run a cluster of Mongrels, i.e. multiple Mongrel instances. These Mongrel instances do not talk to the Internet directly. Instead, they are put behind a load balancer or a web server, which proxies requests to this cluster of Mongrels. If you use Apache on Windows then you can:
1. Setup and start a cluster of Mongrels, each listening on its own port.
2. Setup a virtual host with some mod_proxy_balancer directives, with which you tell mod_proxy_balancer to proxy all requests to the Mongrel cluster. mod_proxy_balancer will automatically distribute the load between the Mongrels.
If usage is really low i.e. likely to be mostly 1 person at a time or your response time is really low then you can get away with a single mongrel and having your users point there browser to the relevent IP address and port.
For some time before I finally got my apps migrated to our corporate Linux/apache "cloud" (which was anything but straightforward, for mostly internal IT-related reasons) I ran two apps on a workstation, using a separate mongrel (different ports) for each. It worked well enough to be useful for almost a year.
These days (well, about three weeks now) I've substituted the immediacy, control - and vulnerability - of local (under my desk) access for the stability of five servers, each with multiple mongrels, staging areas, and deployment annoyances. Swings and roundabouts.