I've got a rather large, 2.3 upgraded to Rails 3 application, that's fat enough it's not making it through the 60 second startup door at Heroku, and therefore it's crashing. I've done a bunch of work to minimize load times within Gems and initializers, but there's some random process that's burning time and I'm not exactly sure what it is. I could use another set of eyes.
Here's a GIST with the config.ru, application.rb, and environment.rb and the Gemfile.
https://gist.github.com/2026140
Any thoughts would be greatly appreciated.
This was due, at least in my case, to two things: 1) a lot of gems, and 2) Mongo taking a long time to initialize (big burdened database).
To fix the gems, on my local dev, I patched bundler Kernel#require statement so I could see which were taking the longest to load. Then, I tried to remove them. Barring that, I set them to :require => false and manually required them where they were needed.
Secondly, I monkey patched Mongoid so that it doesn't try to connect to the database when the app is starting. This helped dramatically with the slow boot time (removed over 10 seconds).
Heroku's boot timeout bit me too. I read several blog posts about how to get around it and ended up automating some of the solutions into a gem.
To reduce the startup time on deploy, you can trim the gems loaded at boot time (this doesn't mean you have to trim them from the app, just boot time).
gem_bench evaluates which gems are likely to not be needed at boot time.
I have an app with about 250 gems and was able to add :require => false to about 60 of them, with dramatic effects.
https://github.com/acquaintable/gem_bench
Disclaimer: I am the author of this open source ruby gem. I wrote the gem to aid myself in solving this exact problem: the 60 second timeout on Heroku.
The heroku-forward gem will help you beat the 60s Heroku timeout.
Related
I have a problem on my Rails application.
I am in version 3.2.22 of rails and 2.2.5 of ruby connect to a mongodb 2.6.
The problem is that I have huge difference in performance on simple or even more complex queries.
For example :
I run rails c development and then I execute my function (quite complex) it responds after 30 seconds
I run rails c production, I perform the same function as the previous one, it responds after 6 minutes 30 seconds, 7 times slower.
So I try to copy pasted the configuration 'development' in 'production', but the result remains the same, same for the Gemfile.
I look in all the code of the project no difference between the environment production and development.
Do you know the differences in the heart of rails between these two environments? did anyone ever encounter the problem?
Importantly, I am of course connecting to the same database.
Thanks in advance.
You have not specified your mongo (Ruby driver) and mongoid versions, if they are old you may need to upgrade and/or adjust the code to your environment.
To determine whether the slowdown happens in the database or in your application, use command monitoring as described here: https://docs.mongodb.com/ruby-driver/current/tutorials/ruby-driver-monitoring/#command-monitoring
Look at the log entries corresponding to your queries and make note of how log they take in each environment. By implementing a custom event subscriber you can also save the commands being sent and verify that they are identical between the two environments.
I got this!
When I saw the number of requests in production, I immediately thought of the query cache.
I found the 'identity_map_enabled' parameter for mongo, so I changed it to true, and hop magic!
I work with several Rails apps, some on Rails 3.2/Ruby 2.0, and some one Rails 2.3/Ruby 1.8.7.
What they have in common is that, as they've grown and added more dependencies/gems, they take longer and longer to start. Development, Test, Production, console, it doesn't matter; some take 60+ seconds.
What is the preferred way to first, profile for what is causing load times to be so slow, and two, improve the load times?
There are a few things that can cause this.
Too many GC passes and general VM shortcomings - See this answer for a comprehensive explanation. Ruby <2.0 has some really slow bits that can dramatically increase load speeds; compiling Ruby with the Falcon or railsexpress patches can massively help that. All versions of MRI Ruby use GC settings by default that are inappropriate for Rails apps.
Many legacy gems that have to be iterated over in order to load files. If you're using bundler, try bundle clean. If you're using RVM, you could try creating a fresh gemset.
As far as profiling goes, you can use ruby-prof to profile what happens when you boot your app. You can wrap config/environment.rb in a ruby-prof block, then use that to generate profile reports of a boot cycle with something like rails r ''. This can help you track down where you're spending the bulk of your time in boot. You can profile individual sections, too, like the bundler setup in boot.rb, or the #initialize! call in environment.rb.
Something you might not be considering is DNS timeouts. If your app is performing DNS lookups on boot, which it is unable to resolve, these can block the process for $timeout seconds (which might be as high as 30 in some cases!). You might audit the app for those, as well.
Ryan has a good tutorial about speeding up tests, console, rake tasks: http://railscasts.com/episodes/412-fast-rails-commands?view=asciicast
I have checked every methods there and found "spring" the best. Just run the tasks like:
$ spring rspec
The time for your first run of spring will be same as before, but the second and later will be much faster.
Also, from my experience, there will be time you need to stop spring server and restart when there is weird error, but the chance is rare.
For ruby 2 apps, try zeus - https://github.com/burke/zeus
1.8 apps seem to boot much faster than 1.9, spork might help? http://railscasts.com/episodes/285-spork
My 3.1.3 rails app takes quite a while to start up, and even running rails console seems to take longer than it reasonably should. For example, with my app it's 50 seconds from rails c to the command prompt. In a test fresh rails app (e.g. from rails new) it's about 5 seconds.
Needless to say, this is really annoying, particularly when trying to run tests, etc.
I've seen the links at https://stackoverflow.com/a/5652640/905282 but they're pretty involved; I was hoping for maybe something that would be at a higher level, like "oh yeah, here's how long each gem is taking up during startup".
Suggestions, or do I just need to dive into the details?
Ruby 1.9.3 fixes a performance problem in 1.9.2 when a large number of files have been loaded with require.
That post describes how the performance of including new files is O(N), getting progressively slower the more files are already loaded. Since Rails loads in a lot of files, it is a serious drag on start-up time.
I'm new to Rails 3.
I use ruby 1.9.2 and Rails 3.0.7 and Windows 7
So, my problem
When I start a server this process last for a minute
When I try to access it from the browser (http://127.0.0.1:3000/demo/index) this page loads very long (from 1 minute and more)
I tried to turn off the antivirus, user faster_require gem... I just have no clue what to do...
Whats is the problem?
When Rails starts up it needs to load the entire stack as well as a good chunk of your application, so this can take some time. It's not abnormal for it to take twenty to thirty seconds to get ready even on a current machine.
Generally this isn't an issue as the framework will do smaller reloads while it is running if in development mode. Anything you change in app/ or config/routes.rb will be detected and adjusted for between requests.
The first page load is always the slowest, but after that you should have a very responsive server. If not, something might be amiss configuration-wise.
Some people suggest using Mogrel instead of webrick. I'd recommend to give it a try.
I had this problem with a non-rails project and Apache. Disabling IPv6 fixed the problem. YMMV.
Anybody know a nice way to restart a mongrel cluster via capistrano in a "rolling" style, eg, one mongrel at a time. Would be great to have a bit of wait time in there as well for each, to let the mongrel load the rails app up as well.
I've done some searching, and haven't found too much, so looking for help before I dive into the mongrel_cluster gem myself.
Thanks!
I agree with the seesaw approach more than the rolling approach you are seeking. The problem is that you end up in situations where load balancing can throw users back and forth between different versions of the application while you are transitioning.
The solutions we came up with (before finding SeeSaw, which we don't use) was to take half of the mongrels off line from the load balancer. Shut them down. Update them. Start them up. Put those mongrels back online in the load balancer and take the other half off. Shut the second half down. Update the second half. Start them up. This greatly minimizes the time where you have two different versions of the application running simultaneously.
I wrote a windows bat file to do this. (Deploying on Windows is not recommended, btw)
It is very important to note that having database migrations can make the whole approach a little dangerous. If you have only additive migrations, you can run those at any time before the deployment. If you are removing columns, you need to do it after the deployment. If you are renaming columns, it is better to split it into a create a new column and copy data into it migration to run before deployment and a separate script to remove the old column after deployment. In fact, it may be dangerous to use your regular migrations on a production database in general if you don't make a specific effort to organize them. All of this points to making more frequent deliveries so each update is lower risk and less complex, but that's a subject for another response.
Seesaw is a gem found in the Rails Oceania Rubyforge Project that provides this kind of functionality to mongrel clusters. However, the project may be suffering from some bit-rot not havain had a release since 2007. Still worth a look even just to pinch the ideas :)
#!/bin/bash
for PIDFILE in /tmp/mongrel.*; do
PID=$(cat ${PIDFILE})
kill ${PID}
${RUN_MONGREL_CMD} ${PID}
sleep 2
done