I have a handful of Ruby/Rails apps I'm building locally. I've noticed lately that no matter which app I'm working on, my Ruby process will build up to 90+%, the CPU fan will kick on, and there's no stopping it unless I kill the sucker. Doesn't seem to matter which app I'm working on. Any ideas? Any way to track what's wrong?
I've run the apps on Mongrel, Apache/Passenger, and POW, and every time I get the same result.
And BTW I don't have any jobs or tasks running constantly. This CPU hogging is happening without the servers being hit.
My system:
Ruby: ruby 1.8.7 (2009-06-12 patchlevel 174) [universal-darwin10.0]
Rails: Rails 3.0.4
OSX 10.6.7, MBP 2.53 GHz Core 2 Duo
Have you tried opening a terminal window and running top, or opening Activity Monitor, then running each one individually and seeing when your CPU starts to climb? Both top and Activity Monitor will let you sort by the CPU load an app is generating. Use top -o cpu and watch to see if the highest load is the app you ran or maybe something else the app is causing to run.
A Rails app will spike when it starts up but should settle down as it waits for incoming connections. If you have periodic tasks it's performing you should see those cause the CPU activity to spike again then drop when the task ends.
You're on a MacBook Pro. How much RAM do you have? Maybe your apps are running low and having to swap out too much? That would affect your overall system performance making the CPU work harder, causing it to heat up. A MacBook Pro's hard disk is designed for battery efficiency, not high performance, so if you're hitting the disk hard with a lot of database I/O, you could be heating the machine up and causing the apps to wait due to record locking or some sort of contention.
There's a lot of different things that can cause your machine and apps to slow down, and you haven't really given us a lot to work with, so those are some general ideas of what I'd look into.
I have noticed this also and was able to pin-point it.. If I run top -o cpu, and then open another terminal window and run IRB, and then close the terminal window without exiting out of IRB, then suddenly top shows a ruby process at around 90% CPU. If I do the same experiment but exit out of IRB prior to closing the window, there is no 90% ruby hog.
So, apparently we need to be careful to not close terminal windows unless we have properly exited out of ruby programs.
This might be more of a suggestion than an answer, because I don't know if it covers your issue, but it takes just a few minutes, and you might want to try it.
I have had this problem on FreeBSD, and here was my solution. Somehow, the versions of Ruby / Rails / etc were not compatible with one another, and they also did not give me error messages I could always chase down.
I'm sure you are doing this, but make sure your bundle is always up-to-date.
I switched to rvm (Ruby Version Manager), and stopped using system Ruby altogether. This was overall the best sysadmin move I made with Ruby, because Mac OS X / Macports (my dev machine) and FreeBSD / ports (my production machine) are way behind in Ruby versions.
I can see that you are at ruby 1.8.7 patchlevel 174 on Mac OS X -- the current version's patchlevel is somewhere at or above 330.
Moreover, if you are pushing to a production server that's not Mac OS X, then you're going to probably get better portability with RVM, because you can install the same versions of Ruby, Rails, and all gems, on both machines.
Related
We are running 2 rails application on server with 4GB of ram. Both servers use rails 3.2.1 and when run in either development or production mode, the servers eat away ram at incredible speed consuming up-to 1.07GB ram each day. Keeping the server running for just 4 days triggered all memory alarms in monitoring and we had just 98MB ram free.
We tried active-record optimization related to bloating but still no effect. Please help us figure out how can we trace the issue that which of the controller is at fault.
Using mysql database and webrick server.
Thanks!
This is incredibly hard to answer, without looking into the project details itself. Though I am quite sure you won't be using Webrick in your target production build(right?), so check if it behaves the same under Passenger or whatever is your choice.
Also without knowing the details of the project I would suggest looking at features like generating pdfs, csv parsing, etc. Seen a case, where generating pdf files have been eating resources in a similar fashion, leaving like 5mb of not garbage collected memory for each run.
Good luck.
I've been developing Ruby on Rails on my Macbook Pro for quite some time, but recently I'm starting to notice, that everything is just taking forever.
Even simple things like rake -T take over 30 seconds to complete.
$ time rake -T
real 0m35.113s
user 0m12.997s
sys 0m1.828s
When I do the same thing on my VPS, which has much less memory and CPU available, it runs about three times as fast. I'm using Ruby 1.9.2 installed via RVM on OS X Lion.
Is there anything that I can check/do to get some instant improvement? I was thinking about upgrading to SDD, but I don't want to make such a decision too early.
It could be some gem's fault, perhaps one that adds a lot of middleware.
This might help you.
Also, please try installing 1.9.3, it should perform better at loading.
Edit
Just wanted to add that, if all else fails, [the Mac counterpart of] good ole strace could be of some help.
I've been using Ruby Enterprise Edition and Passenger (for Apache, since I run Apache anyway for other things) for some time, but I'm wondering if there's a new trend about what to use on servers nowadays.
For example I've heard about Thin, Unicorn... I also know that 1.9.2 is faster than REE, but I wonder about RAM consumption. I'd rather have it consume less RAM even at the expense of some speed.
Thanks for all advice.
If you want minimal memory you should try Thin.
It does not have master worker as Unicorn or Passenger, thus uses less memory.
Suppose you have a very small app that needs to run on a small VM, then you can use 1 thin worker + nginx. I ran several rails 3.2 apps using Thin+nginx+postgres on 256MB VMs without swapping.
Unicorn is faster but it needs a master worker. It's good if you want to run on Heroku, you can set it 2 or 3 workers and be within the 512MB limit.
If your app is very big and you have too many long running requests, I would check out jRuby and Thinidad/Torquebox.
I converted a few apps from MRI+Sidekiq to jruby+Trinidad+Trinidad_Scheduler. I get about 100-200 req/sec using a pool of 50 threads in a trinidad server!
What I like about jRuby is that you can combine everything on one Rails Server. You can put together on the same JavaVM the cache_store with EHcache, Scheduling, Background processing and real multithreading.
You don't need to run redis, memcached, resque or sidekiq separately.
Im not saying they are not good, I love sidekiq and resque, but you can decrease your complexity by combining everything on one process and have high concurrency.
A more advanced and Enterprise solution is Torquebox, it has support for clustering and is super scalable. But I've had problems with my app crashing on torquebox, so i'm sticking to Trinidad for now.
The disadvantages of jRuby? MEmory! A Trinidad server will use minimum 512MB, up to 2-3GB ram.
Also, for Single Thread server, a single request from a rails app running Ruby-1.9.3 is about twice as fast as the same request on jRuby.
Another option is Puma, you can get full multithreading on MRI with puma. I myself could not get it stable enough on my apps.
So, it all depends on your requirements, memory usage, full threading and concurrency.
Apart from Passenger, have a look at Unicorn, Trinidad, Puma and Torquebox. Those seems to be the top rails servers right now.
There is an great book with an introduction of converting your Rails app to jRuby and deploy your app using several methods such as trinidad.
http://pragprog.com/book/jkdepj/deploying-with-jruby
The Torquebox Documentation is amazingly good. It's very detailed and explains really good how to use all Torquebox features.
http://torquebox.org/documentation/
I Hope that sharing my experience has helped.
Passenger is still extremely strong, especially being REE will naturally support 1.9 in the near future. The fact that your application can crash, however it won't affect anything else on your machine is an amazing feature to have. Deploying code is extremely easy because the server will continue to accept connections, which means less frustration/stress for you.
However, in terms of comparisons:
Here is a great resource is check out various comparisons(including memory consumption) with all the new servers.
It compares Thin, Unicorn, Passenger, TorqueBox, Glassfish, and Trinidad:
http://torquebox.org/news/2011/03/14/benchmarking-torquebox-round2/
Mike Lewis' link does a good job of comparing those different ruby servers. My personal experience has been with nginx/REE/Passenger and its been good. I haven't tried the others, so I can't comment on that.
However, I can speak on RAM usage. Your biggest savings of RAM will come from using 32-bit servers. In my experience (3x 3GB app servers), 64-bit REE/passenger processes took up to 2x as much RAM as their 32-bit counterparts. We saw a significant performance increase moving from 64 to 32 bit servers, everything else staying the same. Unless your application requires 64-bit, I would suggest running your application servers (not database) in 32-bit.
Passenger is still a very good choice to use so you are not behind the times or anything. It is also actively supported and has a very good development team that contributes a lot to the community. We have been using Unicorn and it has been very good. Our favorite functionality is to be able to upgrade apps/ruby/nginx without dropping a connection.
I have done a few projects using Ruby on Rails. I am going to use JRuby on Rails and hosting it on GAE. In that case what are the stuff that I need to know while developing JRuby apps. I read that
JRuby has the same syntax
I can access Java libraries
JRuby does not have access to some gems/plugins
JRuby app would take some time to load for the first time, so I have to keep it alive by sending
request every 5 mins or so
I cannot use ActiveRecord and instead I must DataMapper
Please correct if I am wrong about any of the statements I have made and Is there anything else that I must know?. Do I need to start reading about JRuby from the scratch or I can go about as usual developing Ruby apps?
I use JRuby everyday.
True:
JRuby has the same syntax
JRuby does not have access to some gems/plugins
I can access Java libraries
Some gems/plugins have jruby-specific versions, some don't work at all. In general, I have found few problems and as the libraries and platforms have matured a lot of the problems have gone away (JRuby has become a lot better).
You can access Java, but in general why would you want to?
False:
JRuby app would take some time to load for the first time, so I have to keep it alive by sending request every 5 mins or so
I cannot use ActiveRecord and instead I must DataMapper
Although I guess it is possible to imagine a server setup where the initial startup/warmup cost of the JVM means you need to ping the server, there is nothing inherent in JRuby that makes this true. If you need to keep the server alive, you should look at your deployment environment. Something similar happens in shared-hosting with passenger where an app can go out of memory after a period of inactivity.
Also, we use ActiveRecord with no problems at all.
afaik, rails 3 is 100% compatible with jruby, so there should be no problem on that path.
like every new platform, you should make yourself comfortable with it by playing around with jruby. i recommend using RVM to do that.
as far as you questions go:
JRuby is just an other runtime like MRI or Rubinus
since JRuby is within the JVM using Java is very easy, but you can also use RJB from MRI
some gems are not compatible, when they use native c libraries, that do not run on JRuby
the JVM and your application container need startup time and some time to load your app, but that is all, there is no need for keep alive, that is wrong
you can use whatever you want, most gems are updated to be compatible with JRuby
#TobyHede mostly covered issues that you thought of you might have so I'll leave it at that.
As for other things to have in mind, it's simply a different interpreter and funny discrepancies will crop up that will take some adaptation.
some methods are implemented differently, such as sleep 10.seconds will throw exception (you have to sleep 10.seconds.to_i) and I remember getting NoMethodError on Symbol class when switching from MRI to JRuby (don't remember which method wasn't implemented), just have in mind slight variations will be there
you will experience hangs and exceptions in gems that otherwise worked for you (pry for example when listing more then one page)
some gems may work differently, pry (again) will exit if you press ctrl+c for example, pretty annoying
slightly slower load times of everything and no zeus
you'll get occasional java exception stack traces with no indication on which line of ruby code it happened
Timeout.timeout often will not work as expected when its wrapped around net code and stars align badly (this has mostly been fixed in jruby core, but it seems to still be an issue with gems that do their own netcode in pure java)
hidden problems with thread-safety in third party code How do you choose gems for a high throughput multithreaded Rails app? - stay away from EventMachine for example
threads will be awesome (due to nativeness and no gil) and fibers will suck (due to no coroutine support in JVM they're ordinary threads), this is why you often won't get a performance boost with celluloid when compared to MRI
you used to run your rails with MRI Ruby as processes in an OS, you knew how to track their PIDs, bloat, run times, kill them, monitor them etc, this part is not evident when you switch to JRuby because everything has turned to threads in a single process. Java world has very good tools to handle these issues, but its something you'll have to learn
killall -9 ruby doesn't do the trick with jruby when your console hangs (which it does more often then before), you have to ps -ef and then track the proper processes without killing your netbeans etc (minor, but annoying)
due to my last point, knowing Java and the JVM will help you get out of tight spots in certain situations (depending on what you intend to do this may be something you actually really need), choice of deployment server will increase or decrease this need (torquebox for example is a bit notorious for this, other deployment options might be simpler, see http://thenerdings.blogspot.com/2012/09/pulling-plug-on-torquebox-and-jruby-for.html)
...
Also, see what jruby team says about differences, https://github.com/jruby/jruby/wiki/DifferencesBetweenMriAndJruby
But yeah, otherwise its "just the same as MRI Ruby" :)
I've been developing RoR apps for about a month in my Windows Laptop, however everytime that I run the tests it takes forever to finish (10+ minutes).
On a MacBook Pro with similar hardware it runs in a fraction of the time (2 minutes).
Can you guys give me any tips for speeding the whole thing up? or should I consider switching my RoR environment to a Linux/Mac box?
I do most of my ruby development in an Ubuntu VM, just because it is easier to get everything together under linux (I imagine the same applies to OSX). RVM makes my life so much easier.
If your hardware can support it and you don't mind working in a VM, may I suggest you give it a try? VM development has its advantages (namely, taking daily snapshots so you can roll back environment changes) and disadvantages (you lose a bit of speed, but not much these days) but I think it is worth it overall.