I've been trying to debug some super slow performance in running my cucumber features. I've run various calls through ruby-prof and think I see the bottlenecks (not too familiar with using ruby-prof) but do not know the cause or more important the solution. I've include below the output from running rake cucumber.
http://dl.dropbox.com/u/1788885/rake_cucumber.txt
Does anyone have any idea why this is happening or how I could go about debugging it further?
Thanks,
Eric
So, I happen to have been playing around with this all morning. It turns out that if you do:
rake cucumber
that does indeed take forever to run. (About 20 secs on my laptop.) But, apparently:
cucumber
runs just fine w/out the overhead of Rake and it runs in about 8 secs.
I'm not sure of what might be making cucumber run slowly for you. As a possible work around, you could considering using spork. On my Windows 7 netbook, running just one cucumber test went from around 7 minutes to 10 seconds with spork.
Related
We are currently upgrading our Rails 3.2 (Ruby 2, Mongoid 3.1.5) App to Capybara Webkit 1.0.0 from 0.13.1. After the gem upgrade we fixed all new failing specs to comply with Capybara 2's new features and (default) settings. That went quite well. BUT: Our whole test suite is now significantly slower than before (~21 minutes compared to ~12 minutes).
Some tests take about 20 seconds. After lots of debugging we figgured out that the issue is not in those slow tests themselves (they run in 2 seconds as single test or in a selected group) but in the cumulation of several tests. We do run (and test) ajax calls in most of these feature tests. So the guess is that the webkit server gets blocked after some tests. But we didn't have that problem with the old capybara version.
I now, every test suite is quite individual so I don't ask for specifics. I'm happy with any idea which can lead to a solution.
Has anyone experienced (and solved ;-) similar problems? Maybe any ideas I didn't have yet?
Clue: Check for the number of files the webkit server opens and webkit processes during the test run
lsof |grep webkit
I'm using Test::Unit on a large app with a large number of gem dependencies (>75). I'm trying to develop using BDD, but it takes minutes for the app to load it's dependencies before it can run the tests. Is there a way to preload the dependencies and just auto-run the test on changes, or a similar solution?
I would look into Spork. It works wonders.
https://github.com/sporkrb/spork
https://github.com/sporkrb/spork-testunit
I am using RSpec and there's a great tool for it, called Spork. It basically loads your app once and then just reloads modified parts. If you combine it with Guard, you get "continuous testing". That is, you hit 'Save' in your editor and tests start executing, giving you instant feedback. This still amazes me after some months :)
Edit
As #THEM points out, there's a plugin for Spork to support TestUnit. You should look into it.
There was also an interesting article about test speed on the 37Signals blog a while back. Might be of interest even if you end up going with Spork or another solution.
The specs for my rails project have been really slow lately. I did a git bisect to see if I could determine what has been slowing it and I found that certain commits that were previously running just fine are now just as slow as the current HEAD.
This leads me to believe that my problem is being caused by a gem updating or something else that's not under my source control. The problem still occurs on other dev machines so I don't think it's my personal environment either.
What's the best way to track down my slowest tests and then figure out what's slowing them down so much?
This flag will tell you which tests are the bottlenecks:
$ rspec --profile
Check out the test prof gem:
https://test-prof.evilmartians.io/#/?id=recipes
https://github.com/palkan/test-prof
I'm looking to speed up our rails application tests. We're already around 600 tests, it's getting higher and it's starting to take a long time to run.
I know there are some tools to speed up the tests. Some we are already using, some we will start using, and im quite sure some we can't use due to our rails version (2.3.8).
But I came here for a different reason. Few weeks ago, I started using a computer that didn't run our tests for a while, and the tests run really really fast. Instead of taking the usual 20-30 minutes, it was completed within 5-7 minutes, if not less.
At first I thought something is wrong, but the more times I run - it started to become slower and slower until it took me 20-30 minutes to run, again.
Now the tests were the same tests, tools were and are the same. I can't think off anything dramatically that changed, besides the fact that I haven't run the tests for a while (few weeks~) and then run again. Could it be something related to tmp files or any of this sort that can be deleted or tweaked to get our tests to run faster?
Thanks for your help.
Some tips to speed up testing:
Make sure you are using transactional fixtures
Try only initializing instead of persisting objects in the database
put config.whiny_nils = false on your config/environments/test.rb
If you're using Devise, put config.stretches = Rails.env.test? ? 1 : 10 on your config/initializers/devise.rb
Upgrade to 1.9.3
PS: Just saw you're using Rails 2.3.8, so this won't work for you, but I'll leave it here in case someone wants to use:
Put this patch into your application (be sure to remove it when upgrading to 3.2)
When I run my cucumber scenarios as a whole, or with the command: cucumber
I get 7 failing steps. When I run them individually with the work in progress tag they pass fine.
I don't think it's a database state issue.. I'm running with transactions and I also tried running without and cleaning the database with database cleaner.... still does not help.
I tried to run the debugger but it does not seem to work when I run the command cucumber. It only works when I run with the work in progress tag: cucumber -p wip
I thought it might be that things are running too fast and capybara is not checking things properly?
Any ideas?
Eureka! I've been having this same problem for awhile now - my tests got slower and slower the more I added - also, some tests would fail randomly, but only when run as a whole suite - after my tests would finish I would just run the feature again and viola! all passing. Very frustrating - but the MOST frustrating part was the speed - recently I upgraded to snow leopard and compiled everything to 64bits. The result? My tests went from taking 7 minutes to 32!
There's a clue in that however - 64 bit apps use more memory to do do the same thing, apparently - however, when I was running my tests the memory on my machine was never coming close to maxing out. Hint #2? Webrat was going fast, it was only when using culerity/celerity to test javascript that things were really slowing down.
After poking around I found out that jruby tells java to give it a maximum 'heap size' of 512 mbs. JRuby allows you to set java options when it is invoked, and culerity allows an environment variable to invoke jruby any way you like. Sure enough, around that time, java would stop consuming memory and the processor would try to set itself on fire. So are you ready? Here it is:
JRUBY_INVOCATION="jruby -J-Xmx1024m" cucumber
That increased my heap size to a gigabyte and my test time dropped down to 7 minutes! Is that it? Did I get it? I sure hope it helps!