I tried to follow José Valim's advice on faster test suites, but to no avail.
I use Spork and put the AR monkey patch in the Spork.each_run block (see the spec helper below).
However, my request specs fail because the database is not cleaned between runs - specifically, I get errors such as expected 1 record, got X when doing assertions like Model.should have(1).record.
Update
The problem lies with request specs using Javascript. See the following spec - it fails when I use js: true, but not if I remove that (I use RSpec's config.treat_symbols_as_metadata_keys_with_true_values = true, fwiw.)
# encoding: UTF-8
require 'spec_helper'
feature "Create a shift", :js do
scenario %q(
In order to plan the workload
As a coordinator
I want to schedule shifts
And I want to see when they're scheduled for
) do
visit shifts_path
click_link "new_shift_#{Date.current}"
fill_in 'shift_note', with: 'Casper - luk'
click_link_or_button 'submit'
Shift.should have(1).record
Shift.last.begins_at.should == Date.current
page.should have_selector("ol[data-date=\"#{Date.current}\"] li#shift_#{Shift.last.id}")
end
end
I can tell it's related to the DB not being cleaned, because it fails the first time (expected 1 record, got 0), passes the second time (because there's 1 record in the DB) and then fails again on any subsequent runs (expected 1 record, got 2 etc.)
I'm trying to avoid using a gem like DatabaseCleaner, to keep dependencies as few as possible, and to avoid a speed decrease in the test suite.
Any help/info/pointers are greatly appreciated!
Relevant info:
Rails 3.2.2
RSpec 2.9.0
Capybara 1.1.2
Capybara-webkit 0.11.0
FactoryGirl 2.6.4
Spork 0.9.0
Guard 1.0.1
Guard-spork 0.5.2
on a Macbook Air, OS X 10.7.3 (if that's relevant)
And my spec helper:
# encoding: UTF-8
require 'rubygems'
require 'spork'
Spork.prefork do
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
require 'capybara/rspec'
Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}
RSpec.configure do |config|
config.mock_with :rspec
config.use_transactional_fixtures = true
config.treat_symbols_as_metadata_keys_with_true_values = true
config.infer_base_class_for_anonymous_controllers = false
config.include Factory::Syntax::Methods
Capybara.javascript_driver = :webkit
Rails.logger.level = 4 # speed - http://blog.plataformatec.com.br/tag/capybara/
end
end
Spork.each_run do
require 'factory_girl_rails'
class ActiveRecord::Base
mattr_accessor :shared_connection
##shared_connection = nil
def self.connection
##shared_connection || retrieve_connection
end
end
ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection
end
After much investigating, the problem seems to be with specs using JS in general, and not really the AR monkey patch.
I've re-phrased the problem in a new question here: RSpec+Capybara request specs w/ JS not working
Related
guys.
Rails 3.2.12
I have a sample test spec/models/user_spec.rb
require 'spec_helper'
describe User do
pending "add some examples to (or delete) #{__FILE__}"
end
spec/spec_helper.rb
require 'spork'
Spork.prefork do
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
require 'rspec/autorun'
Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }
RSpec.configure do |config|
config.fixture_path = "#{::Rails.root}/spec/fixtures"
config.use_transactional_fixtures = true
config.infer_base_class_for_anonymous_controllers = false
config.order = "random"
config.color_enabled = true
config.tty = true
config.formatter = :documentation # :progress, :html, :textmate
end
end
Spork.each_run do
end
When I run the test without Spork I get:
$ rspec
c:/ruby/Ruby193/lib/ruby/gems/1.9.1/gems/activesupport-3.2.12/lib/active_support
/dependencies.rb:251:in `block in require': iconv will be deprecated in the futu
re, use String#encode instead.
User
add some examples to (or delete) d:/sites/efiling/spec/models/user_spec.rb (PE
NDING: No reason given)
Pending:
User add some examples to (or delete) d:/sites/efiling/spec/models/user_spec.r
b
# No reason given
# ./spec/models/user_spec.rb:4
Finished in 0.31249 seconds
1 example, 0 failures, 1 pending
Randomized with seed 10437
When I run the test with Spork I get:
$ rspec . --drb
<-- Slave(1) run done!
In Spork window:
Loading Spork.prefork block...
--> DRb magazine_slave_service: 1 provided...
Running tests with args ["--color", "."]...
<-- take tuple(1); slave.run...
-- (1);run done
Done.
-- build slave 1...
Preloading Rails environment
Why does Rspec give no info about pending, failed etc tests when I use it with Spork?
I found an answer to your question: How can I configure rspec to show output with spork?
For your convenience, the way posted to circumvent this error is to add this to the Spork.each_run block:
if Spork.using_spork?
RSpec.configure do |config|
config.reset
config.output_stream = $stdout
end
end
I had the same problems as you, and once I added this in, I was getting normal output again.
Am trying to work on coverband gem that will show the code covered in my application. I followed commands from this link https://github.com/danmayer/coverband, and used "rake test" to generate coverage. Added
require 'simplecov'
SimpleCov.start
to test/test_helper.rb and config/application.rb as in simplecov gem documentation. My question is how to get output for the entire clicks and adds that i do on my application? For now i end up with output on initializers and configs but i need codes covered for controllers, models and views.
Would somebody help with this ?
Hey I am the author of Coverband. I wanted to help make sure people understand that Coverband isn't for test code coverage. For that you should just look at Simplecov which has all you need. Coverband, provides runtime code coverage, which is useful for seeing what code is executed on production servers. This helps find features and code that is being maintained but isn't in use by any actual users. The use case from Simplecov is very different. Coverband uses the Simplecov output formatters, but besides that they don't share other code or the same goals.
Hope that helps.
I managed to get a working spec/spec_helper.rb configuration that executes SimpleCov correctly simply using the $ rspec spec/ command thanks to a comment on the Github issue that sent me to this blog entry, and its example spec/spec_helper.rb. All reasons why this works are contained in the (very detailed!) blog entry. Replace SampleApp with the name of your application
Spork.prefork do
unless ENV['DRB']
require 'simplecov'
SimpleCov.start 'rails'
end
require 'rails/application'
require Rails.root.join("config/application")
ENV["RAILS_ENV"] ||= 'test'
require 'rspec/rails'
require 'rspec/autorun'
RSpec.configure do |config|
config.mock_with :rspec
config.fixture_path = "#{::Rails.root}/spec/fixtures"
config.use_transactional_fixtures = false
config.before :each do
if Capybara.current_driver == :rack_test
DatabaseCleaner.strategy = :transaction
else
DatabaseCleaner.strategy = :truncation
end
DatabaseCleaner.start
end
config.after do
DatabaseCleaner.clean
end
config.infer_base_class_for_anonymous_controllers = false
end
end
You might try using Cucumber with a Web driver. That should get clicks in the coverage report. If you need Javascript evaluation, you'll want to use Selenium.
I am using following gems and ruby-1.9.3-p194:
rails 3.2.3
rspec-rails 2.9.0
spork 1.0.0rc2
guard-spork 0.6.1
Full list of used gems is available in this Gemfile.lock or Gemfile.
And I am using this configuration files:
Guardfile
.rspec
spec_helper.rb
factories.rb
If I modify any model (or custom validator in app/validators etc) reloading code doesnt works.
I mean when I run specs (hit Enter on guard console) Spork contain "old code" and I got obsolete error messages. But when I manually restart Guard and Spork (CTRC-C CTRL-d guard) everything works fine. But it is getting tired after few times.
Questions:
Can somebody look at my config files please and fix error which block updating code.
Or maybe this is an issue of newest Rails version?
PS This problem repeats and repeats over some projects (and on some NOT). But I haven't figured out yet why this is happens.
PS2 Perhaps this problem is something to do with ActiveAdmin? When I change file in app/admin code is reloaded.
Workaround:
# config/environments/test.rb
config.cache_classes = false
But it is "double-edged sword".
Specs run now ~2.0x time longer. But it is still faster than restarting again and again Spork.
Update 28.06.2013
Use Zeus. It works perfectly. Benchmarks are at the bottom..
If you are using 1.9.3 consider installing special patches which REALLY speed up loading app.
RVM patchsets
rbenv instructions
Background & Benchmark:
I have a quite large 1.9.3 app and I wanted to speedup app loading, Spork doesn't work so I started looking for other solutions:
I write a empty spec to see how long it takes to load my app
-/spec/empty_spec.rb
require 'spec_helper'
describe 'Empty' do
end
plain 1.9.3
time rspec spec/empty_spec.rb 64,65s user 2,16s system 98% cpu 1:07,55 total
1.9.3 + rvm patchsets
time rspec spec/empty_spec.rb 17,34s user 2,58s system 99% cpu 20,047 total
1.9.3 + rvm patchsets + zeus
time zeus test spec/empty_spec.rb 0,57s user 0,02s system 58% cpu 1,010 total
[w00t w00t!]
Alternatively, you can add guards for your models, controllers and other code. It'll make guard reload spork when any of these files change.
guard 'spork',
:rspec_env => {'RAILS_ENV' => 'test'} do
watch(%r{^app/models/(.+)\.rb$})
watch(%r{^lib/(.+)\.rb$})
end
I had the same problem. Tests were reloaded and ran successfully for changes to model_spec.rb. When I made changes to the model.rb file the tests were re-run, however the code seemed to be cached - so the changed were not applied.
It required a combination of a few answers to get things working:
# /config/environments/test.rb
config.cache_classes = !(ENV['DRB'] == 'true')
# spec_helper.rb
Spork.each_run do
.....
ActiveSupport::Dependencies.clear
end
I also updated spork to (1.0.0rc3) and replaced the spork gem with spork-rails, as mentioned by #23inhouse above. However, I did not see any difference between either gem in the gemfile although upgrading spork may have had an effect.
Hopefully this helps someone else not spend any more hours banging their head against the wall.
Great as Spork is, it seems to break on every Rails upgrade :(
On Rails, 3.2.3, I've added this snippet in spec/spec_helper.rb to forcibly reload all ruby files in the app directory.
Spork.each_run do
# This code will be run each time you run your specs.
Dir[Rails.root + "app/**/*.rb"].each do |file|
load file
end
end
In my case the problem was draper. It didn't allow spork to reload the models.
Spork.prefork do
ENV['RAILS_ENV'] ||= 'test'
# Routes and app/ classes reload
require 'rails/application'
Spork.trap_method(Rails::Application::RoutesReloader, :reload!)
Spork.trap_method(Rails::Application, :eager_load!)
# Draper preload of models
require 'draper'
Spork.trap_class_method(Draper::System, :load_app_local_decorators)
# Load railties
require File.expand_path('../../config/environment', __FILE__)
Rails.application.railties.all { |r| r.eager_load! }
...
Just remember to insert the trap method for Draper before loading the environment.
Spork got cleaned up and some functionality was extraced.
https://github.com/sporkrb/spork-rails
add this to your Gemfile
gem 'spork-rails'
Fixed the same problem by adding more to the spork.each_run method.
Rails 3.2.2
Also, I recommend running one test a time. It's much faster, less noisy, and we normally work on one test at a time anyway.
rspec spec -e 'shows answer text'
I find it is faster and easier than using Guard because I was just sitting around waiting for Guard to finish. Also, Guard did not always reload the right files and run the right tests when I made a change.
spec_helper.rb file:
require 'spork'
Spork.prefork do
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rspec/rails'
require 'rspec/autorun'
require 'capybara/rspec'
require 'capybara/rails'
# Requires supporting ruby files with custom matchers and macros, etc,
# in spec/support/ and its subdirectories.
end
Spork.each_run do
Dir[Rails.root.join('spec/support/**/*.rb')].each {|f| require f}
RSpec.configure do |config|
# config.mock_with :mocha
# config.mock_with :flexmock
# config.mock_with :rr
config.mock_with :rspec
# Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
config.fixture_path = "#{::Rails.root}/spec/fixtures"
# If you're not using ActiveRecord, or you'd prefer not to run each of your
# examples within a transaction, remove the following line or assign false
# instead of true.
config.use_transactional_fixtures = true
# If true, the base class of anonymous controllers will be inferred
# automatically. This will be the default behavior in future versions of
# rspec-rails.
config.infer_base_class_for_anonymous_controllers = false
config.include RequestHelpers, :type => :request
config.before :suite do
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.clean_with :truncation
end
config.before :each do
DatabaseCleaner.start
end
config.after :each do
DatabaseCleaner.clean
end
config.include(MailerHelpers)
config.before(:each) { reset_email }
end
# This code will be run each time you run your specs.
end
I cannot get request specs working when using Javascript.
My specs pass if I run them without Javascript (the page is built to work with or without JS).
Specifically, the specs fail when I do assertions like Post.should have(1).record.
Capybara just doesn't pick up the records from the DB, and the database is not cleaned between runs.
I've tried using DatabaseCleaner with transactional fixtures disabled - the common approach to this, I guess. No dice.
I've also tried (and, would ideally prefer) running without DatabaseCleaner, using transactional fixtures and forcing AR to share the same connection between threads (a patch described by José Valim). Again, no dice.
Additionally, I've also tried switching between Capybara-webkit and Selenium - the issue persists.
I've put up a sample app with just a basic Post scaffold, that replicates the problem: https://github.com/cabgfx/js-specs
There's a spec_helper.rb with transactional fixtures and AR shared connection, and a spec_helper_database_cleaner.rb for the other scenario.
I normally use Spork, but I've disabled it in both spec_helper.rb files, just to eliminate a potential point of failure (in both apps; the "real" one and the sample app).
I develop locally using Pow on a Macbook Air, running OS X 10.7.3 with MRI 1.9.3 thru RVM. (I also tried on 1.9.2).
Hope I'm making sense - any guidance/help/pointers are greatly appreciated!
Matt - thanks a lot for taking time to assist me!
I tried setting it up with your spec_helper, using Selenium as the javascript driver.
The spec still failed - but I could see the correct behavior being executed in Firefox...
Then it dawned on me, that the problem might occur because of Capybara not waiting for AJAX requests to finish.
I then reverted to my initial spec_helper (with Spork and no DatabaseCleaner), and simply used Capybara's wait_until { page.has_content? "text I'm inserting with JS" }.
I updated the sample app, and just added sleep 1 in the request spec, so you can see for yourself. It now works with and without Spork, and the AR monkey patch seems to work perfectly.
I have tried your code with the spec_helper.rb listed below and the test passes. Note that the syntax for triggering database cleaner is a bit different than in your spec_helper_database_cleaner.rb.
We're using this in production, and we've also tried the modification suggested by Jose Valim but it didn't work for us - this did.
require 'rubygems'
require 'spork'
#uncomment the following line to use spork with the debugger
#require 'spork/ext/ruby-debug'
Spork.prefork do
# Loading more in this block will cause your tests to run faster. However,
# if you change any configuration or code from libraries loaded here, you'll
# need to restart spork for it take effect.
# This file is copied to spec/ when you run 'rails generate rspec:install'
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
require 'rspec/autorun'
# Add this to load Capybara integration:
require 'capybara/rspec'
require 'capybara/rails'
include Capybara::DSL
# Requires supporting ruby files with custom matchers and macros, etc,
# in spec/support/ and its subdirectories.
Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}
RSpec.configure do |config|
# ## Mock Framework
#
# If you prefer to use mocha, flexmock or RR, uncomment the appropriate line:
#
# config.mock_with :mocha
# config.mock_with :flexmock
# config.mock_with :rr
# Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
config.fixture_path = "#{::Rails.root}/spec/fixtures"
# If you're not using ActiveRecord, or you'd prefer not to run each of your
# examples within a transaction, remove the following line or assign false
# instead of true.
config.use_transactional_fixtures = false
# If true, the base class of anonymous controllers will be inferred
# automatically. This will be the default behavior in future versions of
# rspec-rails.
config.infer_base_class_for_anonymous_controllers = false
# Include sign_in & sign_out for tests
# config.include Devise::TestHelpers, :type => :controller
# Use database_cleaner to ensure a known good test db state as we can't use
# transactional fixures due to selenium testing
config.before(:suite) do
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
end
end
Spork.each_run do
# This code will be run each time you run your specs.
end
José's suggestion worked for me but not when I used Spork. But adding this to spec_helper.rb did:
Spork.prefork do
RSpec.configure do |config|
# Make it so poltergeist (out of thread) tests can work with transactional fixtures
# http://www.opinionatedprogrammer.com/2011/02/capybara-and-selenium-with-rspec-and-rails-3/#post-441060846
ActiveRecord::ConnectionAdapters::ConnectionPool.class_eval do
def current_connection_id
Thread.main.object_id
end
end
end
end
Source: http://www.opinionatedprogrammer.com/2011/02/capybara-and-selenium-with-rspec-and-rails-3/#post-441060846
I've been using spork all day, and most of the time it is a really great.
However, I am often running into a few problems where I need to restart Spork in order for my tests to pass... and now I'm wondering if it's more trouble than it's worth. I am new at ruby, so sometimes I can't predict if the error is due to a refresh problem, or if the error is due to my unfamiliarity with Ruby and Rails.
What do I need to put into Spork.each_run block so that my validations and other things are refreshed so that I don't have to restart the spork server?
Thanks
EDIT: If you can upgrade to Ruby 2.0 this is your best bet. It is fast enough and will let you work in regular way without the need for tools like Spork, Zeus, and ect. And in essence you won't need anything I wrote below.
If you still need some speed bump while developing check out the Fast Rails Commands cast.
Well yes, you want to reload Spork if you changed environment, initializer or spec_helper files (and for that guard-spork is perfect), but not when you updated one of your classes (models) as this would deny the purpose of tools like spork. I had the very same issue: I could delete all methods in a model, and tests would still pass, because Spork hold "old" model class in memory. Restarting Spork was required.
Reason:
Some plugins cause the model code to be preloaded so some work is required to block that from happening.
https://github.com/sporkrb/spork/issues/37
https://github.com/sporkrb/spork/issues/94
You want to prevent model code preloaded, as this will not "reload" them if you make any changes (like with validations).
Solutions:
Depends from gems that are involved. In my case, I had to deal with Devise and FactoryGirl, but in essence, you do it by using Spork.trap_method as described on wiki: https://github.com/sporkrb/spork/wiki/Spork.trap_method-Jujitsu
Additionally you can run spork -d to get a list of files that are preloaded, it may be helpful to track which gems may be involved in causing this issue.
Example:
Rails 3.0.x + Rspec2 + Spork 0.9.0.rcX + Capybara + Devise + FactoryGirl
# spec/spec_helper.rb
Spork.prefork do
# This file is copied to spec/ when you run 'rails generate rspec:install'
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
require 'capybara/rspec'
require 'capybara/rails'
# set "gem 'factory_girl', :require => false" in Gemfile
require 'factory_girl'
# deal with Devise
require "rails/application"
Spork.trap_method(Rails::Application, :reload_routes!)
require File.dirname(__FILE__) + "/../config/environment.rb"
Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}
RSpec.configure do |config|
config.mock_with :rspec
config.use_transactional_fixtures = false
config.before(:suite) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
# Devise controller test helpers:
config.include Devise::TestHelpers, :type => :controller
end
end
Spork.each_run do
# deal with factory girl
Factory.definition_file_paths = [File.join(Rails.root, 'spec', 'factories')]
Factory.find_definitions
end
Please note that config.cache_classes = true need to be set to true in test environment, otherwise you may get errors from gems like FactoryGirl.
This made my model tests (specs) run quickly, and "reload" them every time I save a file and fire rspec.
EDIT: If you're running on Ruby 1.9.3 you can try out an interesting alternative: Zeus - https://github.com/burke/zeus
Use Guard to reload Spork when you update your classes Guard::Spork allows to automatically & intelligently start/reload your RSpec/Cucumber Spork server(s).
https://github.com/guard/guard-spork
http://flux88.com/2011/04/using-guard-spork-with-mongoid-devise/
From http://www.rubyinside.com/how-to-rails-3-and-rspec-2-4336.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+RubyInside+%28Ruby+Inside%29 :
A minor snafu will remain, though. If
you update app/models/person.rb, the
change won't take effect in your tests
since Spork has the old Person still
in memory. One way around this is to
edit config/environments/test.rb and
change:
config.cache_classes = true
To:
config.cache_classes = false
With more recent versions of Factory Girl, you don't need to do much. First, add FactoryGirl.reload in Spork.each_run. If you have factories with the class parameter, they should be string.
factory :my_model, class: 'MyModel' do...
instead of
factory :my_model, class: MyModel do...