I work with several Rails apps, some on Rails 3.2/Ruby 2.0, and some one Rails 2.3/Ruby 1.8.7.
What they have in common is that, as they've grown and added more dependencies/gems, they take longer and longer to start. Development, Test, Production, console, it doesn't matter; some take 60+ seconds.
What is the preferred way to first, profile for what is causing load times to be so slow, and two, improve the load times?
There are a few things that can cause this.
Too many GC passes and general VM shortcomings - See this answer for a comprehensive explanation. Ruby <2.0 has some really slow bits that can dramatically increase load speeds; compiling Ruby with the Falcon or railsexpress patches can massively help that. All versions of MRI Ruby use GC settings by default that are inappropriate for Rails apps.
Many legacy gems that have to be iterated over in order to load files. If you're using bundler, try bundle clean. If you're using RVM, you could try creating a fresh gemset.
As far as profiling goes, you can use ruby-prof to profile what happens when you boot your app. You can wrap config/environment.rb in a ruby-prof block, then use that to generate profile reports of a boot cycle with something like rails r ''. This can help you track down where you're spending the bulk of your time in boot. You can profile individual sections, too, like the bundler setup in boot.rb, or the #initialize! call in environment.rb.
Something you might not be considering is DNS timeouts. If your app is performing DNS lookups on boot, which it is unable to resolve, these can block the process for $timeout seconds (which might be as high as 30 in some cases!). You might audit the app for those, as well.
Ryan has a good tutorial about speeding up tests, console, rake tasks: http://railscasts.com/episodes/412-fast-rails-commands?view=asciicast
I have checked every methods there and found "spring" the best. Just run the tasks like:
$ spring rspec
The time for your first run of spring will be same as before, but the second and later will be much faster.
Also, from my experience, there will be time you need to stop spring server and restart when there is weird error, but the chance is rare.
For ruby 2 apps, try zeus - https://github.com/burke/zeus
1.8 apps seem to boot much faster than 1.9, spork might help? http://railscasts.com/episodes/285-spork
Related
I always have a significant delay (c.10 seconds) when running tasks like:
rails server
rake assets:precompile
rspec spec
The delay is at the point before it says:
Connecting to database specified by database.yml
I've found several articles identifying 10 or more different parameters that can be tuned to speed up rake and / or rspec runs. But I'm reluctant to start fiddling around with 10 different parameters without a better understanding of what's causing it.
Is there way to diagnose what's causing that delay? Like something more verbose?
Supplementary info:
Ubuntu 14
Ruby 1.9.3
Rails 3.2
Thanks to #MaxWilliams for the link to this post How do I debug a slow rails app boot time?
I started using Mark Ellul's Bumbler - http://github.com/mark-ellul/Bumbler
It gave me exactly what I wanted - an insight into what's going in the background and which gems are taking the time. Of course I still need to speed up the slow ones (fog and authlogic seem to be two of the main culprits). But that's at different question.
I've got a rather large, 2.3 upgraded to Rails 3 application, that's fat enough it's not making it through the 60 second startup door at Heroku, and therefore it's crashing. I've done a bunch of work to minimize load times within Gems and initializers, but there's some random process that's burning time and I'm not exactly sure what it is. I could use another set of eyes.
Here's a GIST with the config.ru, application.rb, and environment.rb and the Gemfile.
https://gist.github.com/2026140
Any thoughts would be greatly appreciated.
This was due, at least in my case, to two things: 1) a lot of gems, and 2) Mongo taking a long time to initialize (big burdened database).
To fix the gems, on my local dev, I patched bundler Kernel#require statement so I could see which were taking the longest to load. Then, I tried to remove them. Barring that, I set them to :require => false and manually required them where they were needed.
Secondly, I monkey patched Mongoid so that it doesn't try to connect to the database when the app is starting. This helped dramatically with the slow boot time (removed over 10 seconds).
Heroku's boot timeout bit me too. I read several blog posts about how to get around it and ended up automating some of the solutions into a gem.
To reduce the startup time on deploy, you can trim the gems loaded at boot time (this doesn't mean you have to trim them from the app, just boot time).
gem_bench evaluates which gems are likely to not be needed at boot time.
I have an app with about 250 gems and was able to add :require => false to about 60 of them, with dramatic effects.
https://github.com/acquaintable/gem_bench
Disclaimer: I am the author of this open source ruby gem. I wrote the gem to aid myself in solving this exact problem: the 60 second timeout on Heroku.
The heroku-forward gem will help you beat the 60s Heroku timeout.
My 3.1.3 rails app takes quite a while to start up, and even running rails console seems to take longer than it reasonably should. For example, with my app it's 50 seconds from rails c to the command prompt. In a test fresh rails app (e.g. from rails new) it's about 5 seconds.
Needless to say, this is really annoying, particularly when trying to run tests, etc.
I've seen the links at https://stackoverflow.com/a/5652640/905282 but they're pretty involved; I was hoping for maybe something that would be at a higher level, like "oh yeah, here's how long each gem is taking up during startup".
Suggestions, or do I just need to dive into the details?
Ruby 1.9.3 fixes a performance problem in 1.9.2 when a large number of files have been loaded with require.
That post describes how the performance of including new files is O(N), getting progressively slower the more files are already loaded. Since Rails loads in a lot of files, it is a serious drag on start-up time.
I have done a few projects using Ruby on Rails. I am going to use JRuby on Rails and hosting it on GAE. In that case what are the stuff that I need to know while developing JRuby apps. I read that
JRuby has the same syntax
I can access Java libraries
JRuby does not have access to some gems/plugins
JRuby app would take some time to load for the first time, so I have to keep it alive by sending
request every 5 mins or so
I cannot use ActiveRecord and instead I must DataMapper
Please correct if I am wrong about any of the statements I have made and Is there anything else that I must know?. Do I need to start reading about JRuby from the scratch or I can go about as usual developing Ruby apps?
I use JRuby everyday.
True:
JRuby has the same syntax
JRuby does not have access to some gems/plugins
I can access Java libraries
Some gems/plugins have jruby-specific versions, some don't work at all. In general, I have found few problems and as the libraries and platforms have matured a lot of the problems have gone away (JRuby has become a lot better).
You can access Java, but in general why would you want to?
False:
JRuby app would take some time to load for the first time, so I have to keep it alive by sending request every 5 mins or so
I cannot use ActiveRecord and instead I must DataMapper
Although I guess it is possible to imagine a server setup where the initial startup/warmup cost of the JVM means you need to ping the server, there is nothing inherent in JRuby that makes this true. If you need to keep the server alive, you should look at your deployment environment. Something similar happens in shared-hosting with passenger where an app can go out of memory after a period of inactivity.
Also, we use ActiveRecord with no problems at all.
afaik, rails 3 is 100% compatible with jruby, so there should be no problem on that path.
like every new platform, you should make yourself comfortable with it by playing around with jruby. i recommend using RVM to do that.
as far as you questions go:
JRuby is just an other runtime like MRI or Rubinus
since JRuby is within the JVM using Java is very easy, but you can also use RJB from MRI
some gems are not compatible, when they use native c libraries, that do not run on JRuby
the JVM and your application container need startup time and some time to load your app, but that is all, there is no need for keep alive, that is wrong
you can use whatever you want, most gems are updated to be compatible with JRuby
#TobyHede mostly covered issues that you thought of you might have so I'll leave it at that.
As for other things to have in mind, it's simply a different interpreter and funny discrepancies will crop up that will take some adaptation.
some methods are implemented differently, such as sleep 10.seconds will throw exception (you have to sleep 10.seconds.to_i) and I remember getting NoMethodError on Symbol class when switching from MRI to JRuby (don't remember which method wasn't implemented), just have in mind slight variations will be there
you will experience hangs and exceptions in gems that otherwise worked for you (pry for example when listing more then one page)
some gems may work differently, pry (again) will exit if you press ctrl+c for example, pretty annoying
slightly slower load times of everything and no zeus
you'll get occasional java exception stack traces with no indication on which line of ruby code it happened
Timeout.timeout often will not work as expected when its wrapped around net code and stars align badly (this has mostly been fixed in jruby core, but it seems to still be an issue with gems that do their own netcode in pure java)
hidden problems with thread-safety in third party code How do you choose gems for a high throughput multithreaded Rails app? - stay away from EventMachine for example
threads will be awesome (due to nativeness and no gil) and fibers will suck (due to no coroutine support in JVM they're ordinary threads), this is why you often won't get a performance boost with celluloid when compared to MRI
you used to run your rails with MRI Ruby as processes in an OS, you knew how to track their PIDs, bloat, run times, kill them, monitor them etc, this part is not evident when you switch to JRuby because everything has turned to threads in a single process. Java world has very good tools to handle these issues, but its something you'll have to learn
killall -9 ruby doesn't do the trick with jruby when your console hangs (which it does more often then before), you have to ps -ef and then track the proper processes without killing your netbeans etc (minor, but annoying)
due to my last point, knowing Java and the JVM will help you get out of tight spots in certain situations (depending on what you intend to do this may be something you actually really need), choice of deployment server will increase or decrease this need (torquebox for example is a bit notorious for this, other deployment options might be simpler, see http://thenerdings.blogspot.com/2012/09/pulling-plug-on-torquebox-and-jruby-for.html)
...
Also, see what jruby team says about differences, https://github.com/jruby/jruby/wiki/DifferencesBetweenMriAndJruby
But yeah, otherwise its "just the same as MRI Ruby" :)
I am looking for a system to parallelise a large suite of tests in a Ruby on Rails app (using rspec, cucumber) that works using JRuby. Cucumber is actually not too bad, but the full rSpec suite currently takes nearly 20 minutes to run.
The systems I can find (hydra, parallel-test) look like they use forking, which isn't the ideal solution for the JRuby environment.
We don't have a good answer for this kind of application right now. Just recently I worked on a fork of spork that allows you to keep a process running and re-run specs or features in it, provided you're using an app framework that supports code reloading (like Rails). Take a look at the jrubyhub application for an example of how I use Spork.
You might be able to spawn a spork instance for your application and then send multiple, threaded requests to it to run different specs. But then you're relying on RSpec internals to be thread-safe, and unfortunately I'm pretty sure they're not.
Maybe you could take my code as a starting point and build a cluster of spork instances, and then have a client that can distribute your test suite across them. It's not going to save memory and will still take a long time to start up, but if you start them all once and just re-use them for repeated runs, you might make some gains in efficiency.
Feel free to stop by user#jruby.codehaus.org or #jruby on freenode for more ideas.