I'm looking for the quickest way to run unit tests for a rails app on a Windows machine, preferably automatically. My environment is:
Ruby 1.8.7
Rails 3.0.9
ZenTest 3.6.0 (the latest versions 4.6.2/4.5.0 failed when I tried them for some reason)
Currently they run very slowly, eg. 30s to run a suite of 12 very simple unit tests, time mostly spent starting ruby it seems. The tests themselves take 5s to run according to autotest. For someone used to running 100s of tests in 10s, this is agony, and makes TDD infeasible. I'd even be happy if I could re-run one unit test in less than 5s...
I've searched other questions. Some are old and some conflict. What's the latest accepted wisdom on this? Here are the suggestions I'm aware of:
Use faster_require and/or faster_gem_script (though I had problems getting this working...)
Try JRuby (though that seems as slow starting?)
Upgrade ruby to 1.9.x
spork?
doze?
rails-dev-boost?
Getting a Linux box (or VMware) is out of the question at present, though getting more tempting...
You may want to look at something like spork (and a blog entry).
I write my tests using rspec and I have had great success making my tests run much faster with spork. The reasons tests rails tests run so slowly is because of the amount of time it takes to load rails and all the other gems that you use in your app.
If you can also upgrade to ruby 1.9.2 that would be quite useful.
Related
pretty new to all this and ran into a real ditch. I had ruby 2.0.0p353 running with rails 4.1+, everything was setup with homebrew, xcode, git,heroku etc.... I'm on OSX 10.9.4
then came time to try out S3 and install the aws-sdk gem. I was unable to install the nokogiri gem after scouring stackoverflow for days to no avail. I then came across this article online that suggested to update rails and ruby versions. In my attempt to upgrade my ruby version to the latest, I followed the instructions given here as follows:
in terminal and in my app folder:
/usr/local/opt/ruby/bin
export PATH="/usr/local/opt/ruby/bin:$PATH"
source ~/.bash_profile
gem update --system
..I subsequently tried bundle update / installs as well... and receive this error shown on pastebin: http://pastebin.com/Q64j0LwD
Now, things are completely messed up as I am unable to even run a rails server. Getting this error Could not find pg-0.17.1 in any of the sources
I currently have Ruby 2.1.2p95 installed. I don't know what else is installed during this trial and error probably several versions of many things unfortunately. Any advice would be great.
: Could not find pg-0.17.1 in any of the sources
Try postgresapp and use documentation for install and configure postgresql server.
The easiest way to get started with PostgreSQL on the Mac
You're clearly new to rails coding. I made a lot of mistakes when I started. I still make a lot of mistakes, but they're more complex mistakes now, mostly ;)
I'd start by using "RailsInstaller". Excellent way to get all the pieces... except Postgres. As suggested by someone with a name consisting of a lot of non-ASCII characters, PostgresApp is incredibly useful to get a more up to date Postgres without messing with compiling stuff outside the project.
You probably should use "rvm" or "rbenv" to manage your rubies. I've tried both, and I favour rvm. rbenv is recommended, but I spent an awkward half day trying to disentangle the consequence of typing "bundle exec rails new app", before I decided to go back to rvm. rvm works more naturally for me, at the expense of creating gemsets for each project. Which, personally, I like. I want to isolate each thing I add to an OS, so I can work on multiple projects with less contamination, and without having to create a VM for each project.
Contamination is, I think the problem with Nokogiri. Everyone tells you to go install brew or Macports or build it all for yourself from scratch. When you do, you get lots of stuff installed. And that stuff affects the compilation process. I'm about 80% certain that I can't compile Nokogiri with system libraries because other projects have dropped include files that rename iconv_open to libiconv_open - which isn't in the native library. I have Nokogiri compiling now, but only as a result of using GNU iconv, and using the arcane "bundle config" to set up a build time dependency for nokogiri alone to use the /opt/local/include version of iconv. That was time consuming.
So, some advice that I haven't taken for myself yet. Clean out the brew/ports stuff. I suspect that you can run anything you need to, in locally installed project directories, rather than interfering with the OS. Unless you really need something that doesn't ship with a Mac, like an up to date ruby (solved by rbenv/rvm)
Look into Heroku. Low cost way to get started with publishing your small apps.
Make sure you have GitHub and BitBucket accounts. Personally I contribute to projects on GitHub, but my private stuff is on BitBucket - pricing models.
Git looks mad. But git is wonderful. Learn to branch. Lots. Learn to merge. Learn to tag, and learn how to push and pull from an upstream repo. These words may mean nothing to you now, but they will save your sanity. You'll use git to push your projects to Heroku. Just freaking amazing. Learn how to have a staging branch and a live branch, and push each to a different Heroku instance, so you can be testing user acceptance on a public facing server, without contaminating dev or live versions. Git/Heroku. Joyful.
Watch out for several things that bit me... Ruby gets lots of patches. They are meaningful. I spent days trying to work out why a piece of ruby code failed, only to discover it worked in a different patch level. Watch for the updates and apply them - except for 'bundle update'. Don't do 'bundle update' until you are old and wise.
Gem versions - that also bit me. Got a project that worked. Then it didn't, with no code changes at all... except that I'd updated my gems. A later version of a gem upset the code. So...
Bundler is your other friend. Lock down the versions of gems that you need for a project. Don't use "bundle update" unless you are prepared for strange things to happen. Make nice Gemfiles.
You probably need to get to grips with TDD and preferably BDD.
So next thing you need is to get the Qt library installed, and use "gem 'capybara-webkit'" and Cucumber with Rspec-Rails to help you write tests that the browser will execute. Butt saver central, if you start changing gem versions. At least you know when the tests stopped working, and can use git to revert to a known working chunk. More importantly, it saves, eventually, a lot of tedious checking around when something unexpectedly stops working.
Also... make sure that your development group of gems (in Gemfile) includes "better_errors" and "binding_of_caller". A REPL in the browser pane when your code fails, is wonderful.
If you want to just throw some stuff together, e.g. office admin projects that you don't want to spend a load of time refining the UI on, but just build something that works. Try 'hobo'. I find it very useful for rapidly building something. Faster to code it than to spec it or draw it. Seriously. And it is all over-rideable, though I've never turned any Hobo code into something for high scaled usage...
Welcome to the amazing world of developing in Rails, on a Mac. It's a rapidly evolving hoot. Hope that helps. :)
Has anyone seen this problem before? Sometimes when running specs for my Rails 3.2.14 project rspec seems to finish as usual:
Finished in 1.27 seconds
6 examples, 2 failures
Failed examples:
rspec ./spec/models/my_spec.rb:123 # Hello world 1
rspec ./spec/models/my_spec.rb:234 # Hello world 2
but then it just hangs there and won't let me continue working in that shell. I can kill -9 the process from another terminal tab, or just start a new shell and run the tests again there, but it makes test driven development a huge pain.
When I restart my computer, the problem goes away for a while, but it always happens again eventually. After it hangs once, it keeps hanging every time I run rspec, even if I run different tests in a different project. The same tests in the same projects pass just fine on my coworkers' computers every time.
I'm not sure what information would help to answer this question so let me know if there is something I should add to this post. I'm running ruby 2.0.0p195 and rails 3.2.14. I've got Mac OS 10.7.5. I use zsh and rbenv.
Thanks for reading!
I had the same issue and solved it by adding the following to my spec_helper.rb:
SimpleCov.start do
use_merging false
end
Note that I'm running on a virtual box vm with a synched directory.
Here's the explanation I found as to why this works:
"When simplecov exists, by default it will try to merge the recorded coverage with what's on disk. To avoid corruption it uses a lock file to guard this merge. Because my virtual box shared fs is not actually posix compliant, the lock file would never acquire, and silently block forever here.
Since I don't care about merging coverage results, the solution for me was to simply disable this behavior with the use_merging flag."
https://gist.github.com/k-yamada/3930916
Ok, figured it out with a friend's help. All it took was removing SimpleCov from my spec_helper. Not sure why SimpleCov was causing the issue, but I'll post an update here if I find out.
I have followed this tutorial on speeding up rspec with spork, and I am on a win7 x64 box with ruby 1.9.2 and rails 3.2.5. Everything is working, but test still execute slowly. Does spork simply not do much on windows because the OS doesn't support forking?
Is there anything else I can do to speed things up?
I also found this similar SO question, and watched the video by Corey Haines on fast testing. I enjoyed the video, but I can't help feeling that something is off when the state of our tools (slow tests due to rails startup time, in this case) dictates how we structure our code. If that slow startup time didn't exist, would there be any need for his methods? On the other hand, with tests taking 10-30 seconds to run, so many of the benefits of TDD are lost that I see his point of view as well.
In case it's relevant, here's the console output from spork as the rspec was executed a couple times:
$ bundle exec spork
Using RSpec
-- Starting to fill pool...
Wait until at least one slave is provided before running tests...
** CTRL+BREAK to stop Spork and kill all ruby slave processes **
Spork is ready and listening on 8989!
-- Rinda Ring Server listening for connections...
-- build slave 1...
Preloading Rails environment
-- build slave 2...
Preloading Rails environment
Loading Spork.prefork block...
Loading Spork.prefork block...
Running tests with args ["--color"]...
--> DRb magazine_slave_service: 1 provided...
--> DRb magazine_slave_service: 2 provided...
<-- take tuple(2); slave.run...
-- (2);run done
Done.
-- build slave 2...
Preloading Rails environment
Loading Spork.prefork block...
Running tests with args ["--color"]...
<-- take tuple(1); slave.run...
-- (1);run done
Done.
-- build slave 1...
Preloading Rails environment
Loading Spork.prefork block...
--> DRb magazine_slave_service: 2 provided...
The Code Shop is building an MRI Ruby optimized for Windows, you can find more about it on their Website or their Github Repo.
I also suggest you to watch this talk about developping rails apps on Windows
Try checking out http://railscasts.com/episodes/413-fast-tests. This shows a lot of different tools that can improve the speed of your test suite significantly!
Before, I was as patient as anybody else in running RSPEC tests using Windows! Doing rake(s) takes too much of my time and it wasn't really healthy anymore. Deliverables were some kind of delayed because the development in Windows was such a pain. And that's the truth. That's why I switched to Linux. But sometimes, there were still trouble(s) in using Linux (hassle installation of some stuffs and more). I just remained patient until I switched to MAC which is a lot better.
If you are really consistent in using Windows for ROR then running tests would be that slow if there are plenty of modules to test.
I'm kinda sure also that Selenium testing would be a disaster in Windows.
But, you may also try to add some other stuffs like using GUARD (for faster execution of test scripts) wherein you don't have to type rspec spec repeatedly.
See: https://github.com/guard/guard
For the spork, well I also encountered a bug about it (before)... wherein I'm testing some spec files using Linux and then it was so slow that I really hated using it.
And that's the reality.
Check out how I configured SPORK to work for rspec:
spec_helper.rb
See: https://github.com/xirukitepe/animelist/blob/master/spec/spec_helper.rb
I would use a linux VM for this kind of thing...
The biggest increase in test speed I've managed to get with RSpec has been to ensure it never hits the database unless it absolutely has to.
I want to start contributing to Rails, fixing patches, submitting my own code etc, and the Rails guide states that the tests MUST run. However, they're currently not and I'm not quite sure what to do.
I'm running Mac OS X, Ruby 1.8 and I have all the needed gems installed - what can I do?
The best place to look is the Rails continuous integration server, which will tell you if there are any currently failing tests. It looks like the most recent edge Rails build failed, and if you checked out the code at any time between August 8th and 16th, you probably got a build with a few failures.
The failing CI tests were due to errors with SQLite2 (now removed from master) and Postgres. We fixed these, but none of these run from a normal rake test so your problems probably aren't related.
Make sure you have the latest Mocha gem installed and that you follow the directions provided by Mike Gunderloy on his afreshcup.com blog: http://afreshcup.com/2008/10/27/contributing-to-rails-step-by-step/
Whenever I run rspec tests for my Rails application it takes forever and a day of overhead before it actually starts running tests. Why is rspec so slow? Is there a way to speed up Rails' initial load or single out the part of my Rails app I need (e.g. ActiveRecord stuff only) so it doesn't load absolutely everything to run a few tests?
I definitely suggest checking out spork.
http://spork.rubyforge.org/
The railstutorial specifically addresses this, and gives a workaround to get spork running nicely in rails 3.0 (as of this moment, spork is not rails 3 ready out of the box). Of course, if you're not on rails 3.0, then you should be good to go.
The part of the tutorial showing how to get spork running in rails 3.0
http://railstutorial.org/chapters/static-pages#sec:spork
Checking when spork is rails 3.0 ready
http://www.railsplugins.org/plugins/440-spork
You should be able to to speed up your script/spec calls by running script/spec_server in a separate terminal window, then adding the additional -X parameter to your spec calls.
Why is rspec so slow? because it loads all the environement, loads fixtures and all that jazz.
Is there a way to speed up Rails' initial load you could try using mocks instead of relying on the database, this is actually correct for unit testing and will definitly speed up your unit tests. Additionnaly using the spec server as mentionned by #Scott Matthewman can help, same with the autotest from zentest mentionned by #Marc-Andre Lafortune
Is there a way to single out the part of my Rails app I need (e.g. ActiveRecord stuff only) so it doesn't load absolutely everything to run a few tests? what about this
rake test:recent
I am not sure how the rspec task integrate with this but you could definitely use the test:recent task as a template to do the same with rspec tests if the.
rake test:rspec:recent
doesn't exist yet
because it loads all the environement, loads fixtures and all that jazz.
The real culprit is if you run it using rake spec, it runs the db:test:prepare task.
This task drops your entire test database and re-creates it from scratch. This seems ridiculous to me, but that's what it does (the same thing happens when you run rake:test:units etc).
You can easily work around this using the spec application which rspec installs as part of the rspec gem.
Like this:
cd railsapp
spec spec # run all specs without rebuilding the whole damn database
spec spec/models # run model specs only
cd spec
spec controllers/user* # run specs for controllers that start with user
I think the "zen" experience you're looking for is to run spec_server and autospec in the background, with the result being near-instant tests when you save a file.
However, I'm having problems getting these two programs to communicate.
I found an explanation here:
I've noticed that autotest doesn't send commands to the spec_server.
Instead it reloads the entire Rails environment and your application's
plugins everytime it executes. This causes autotest to run
significantly slower than script server, because when you run the
script/spec command the specs are sent to the spec_server which
already has your Rails environment fired up and ready to go. If you
happen to install a new plugin or something like that, then you'll
have to restart the spec_server.
But, how do we fix this issue? I'm guessing it would involve downloading ZenTest and changing code for the autotest program, but don't have time to try it out right now.
Are you running this over Rails? If so, it's not RSpec's initialization that's slow, it's Rails'. Rails has to initialize the entire codebase and yours before running the specs. Well, it doesn't have to, but it does. RSpec runs pretty fast for me under my small non-rails projects.
Running tests can be really slow because the whole rails environment has to load (try script/console) and only then can all tests run. You should use autotest which keeps the environment loaded and will check which files you edit. When you edit and save a file, only the tests that depend on these will run automatically and quickly.
If you're using a Mac I recommend using Rspactor over autotest as it uses a lot fewer resources for polling changed files than autotest. There is both a full Cocoa version
RSpactor.app
or the gem version that I maintain at Github
sudo gem install pelle-rspactor
While these don't speed up individual rspec tests, they feel much faster as they auto run the affected spec's within a second of you hitting save.
As of rspec-rails-1.2.7, spec_server is deprecated in favor of the spork gem.
The main reason is that require takes forever on windows, for some reason.
Tips for speedup:
spork now works with windows, I believe.
You can try "faster_require" which caches locations:
http://github.com/rdp/faster_require
GL.
-rp
If you are on a Windows environment then there is probably little you can do as Rails seems to startup really slowly under Windows. I had the same experience on Windows and had to move my setup to a Linux VM to make it really zippy (I was also using autotest).