I would like to increase the speed of my tests.
Should I use use_transactional_fixtures or go with the database_cleaner gem?
Which database_cleaner strategy is the best? I noticed that after migration from :truncation to :transaction my more than 800 examples run about 4 times faster!
Should I turn off use_transactional_fixtures when I use database_cleaner :transaction?
Is it true that the best strategy for rack_test is :transaction?
What is the best practices for changing strategy on the fly from :transaction to :truncation when using selenium or akephalos?
P.S. Mysql, Rails 3, Rspec2, Cucumber
P.P.S. I know about spork and parallel_test and using them. But they are offtopic. For example, Spork save about 15-20 sec on whole suite run, but changing from :transaction to :truncation dramatically increase running time from 3.5 to 13.5 minutes (10 minutes difference).
1., 2. & 4., You should use transactions (either with use_transactional_fixtures or transactions support from the database_cleaner gem) if you are using capybara's default engine, rack_test. As you noted, using transactions are substantially faster than using a truncation strategy. However, when database writes can go through different threads (as with selenium) transactions won't work. So you'll need to use truncation (or force everything to go through one db thread--another option).
3. Yes, you should turn off use_transactional_fixtures when using the database_cleaner gem since the gem natively support transactions. If you only need transactions then just use_transactional_fixtures and never load the database_cleaner gem.
5. The following code will switch between :transaction and :truncation on the fly. (Tested this with rspec, capybara, rails3.)
Features This should give you the best of both worlds. The speed of rack_test when you don't need to test javascript stuff and the flexibility of selenium when you do.
Also this code takes care of repopulating seed data in cases where it is needed (this method assumes you use seeds.rb to load your seed data--as is the current convention).
Add the following code to spec_helper.
config.use_transactional_fixtures = false
RSpec.configure do |config|
config.before(:suite) do
require "#{Rails.root}/db/seeds.rb"
end
config.before :each do
if Capybara.current_driver == :rack_test
DatabaseCleaner.strategy = :transaction
else
DatabaseCleaner.strategy = :truncation
end
DatabaseCleaner.start
end
config.after(:each) do
if Capybara.current_driver == :rack_test
DatabaseCleaner.clean
else
DatabaseCleaner.clean
load "#{Rails.root}/db/seeds.rb"
end
end
end
Thanks Jo Liss for pointing the way.
PS: How to switch drivers on the fly
The above solution assumes you already know how to switch drivers on the fly. In case some who come here don't, here's how:
As above let's assume that you normally will use the default capybara driver rack_test, but need to use selenium to test some Ajaxy stuff. When you want to use the selenium driver use :js => true or #javascript for Rspec or cucumber respectively. For example:
Rspec example:
describe "something Ajaxy", :js => true do
Cucumber example:
#javascript
Scenario: do something Ajaxy
Using transactional fixtures will be faster since the DBMS doesn't commit changes (and therefore no heavy IO occurs resetting the database between tests) but as you know won't always work.
We have had some success using SQLite in-memory databases in the test environment so tests run super fast while leaving transactional fixtures off. This option is also available for MySQL (use :options to set "ENGINE=MEMORY") but I've never done it personally and if you search you'll find a few threads about caveats involved. Might be worth a look. Depending on your testing methodology it may not be acceptable to use a different DB engine though.
I suggest you enable transactional fixtures and use the DatabaseCleaner gem to selectively disable transactional fixtures per example group. I can't say that I've tried this but since you didn't have any answers I figured anything might potentially help you out.
before(:all) do
DatabaseCleaner.strategy = :transaction
DatabaseCleaner.clean_with(:truncation)
end
before(:each) do
DatabaseCleaner.start
end
after(:each) do
DatabaseCleaner.clean
end
If it were me I'd factor this out into a helper and call it as a one-line macro from each example group that needs transactional fixtures turned off.
Seems like there really should be a better way, though.... best of luck.
RSpec.configure do |config|
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each, :js => true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
end
This is from Avdi Grimm's post about database cleaner and Rspec. Step-by-step analysis of the code is in the article.
Have you used Spork ? It greatly enhances speed.
Related
I'm new in rails. In my project i use rspec + capybara + poltergeist + selenium + database_cleaner.
Source code github link
I have postgresql 9.5.5, ubuntu 16.04 LTS.
When run test
rspec spec/controllers # everything work fine
When run
rspec spec/features # everything work fine too
But when i run all tests
rspec # part of tests fail
When run first acceptance test with :selenium -> question_id in browser 28, but must be 1, because it use database_cleaner.
Why database_cleaner not clean my database? What i do wrong? I spent a day to find a solution, but not found nothing.
Help me please.
P.S. It's a training project.
My Database_Cleaner configuration is:
config.use_transactional_fixtures = false
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each, js: true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
Warden.test_reset!
if Rails.env.test?
FileUtils.rm_rf(Dir["#{Rails.root}/public/uploads"])
end
end
Cleaning the tables means it deletes all records, it doesn't necessarily mean it resets the index counters - just like if you delete record 3 and then add a new record - the new one would be 4. So in your case it's possible 27 records have been deleted and the next one you create would be 28 even though there's only 1 actual record. Query the database to see how many actual records are there to verify that.
Next, your database cleaner config should be more like the recommended config - https://github.com/DatabaseCleaner/database_cleaner#rspec-with-capybara-example . append_after vs after is important for stability and checking the driver name vs just the :js metadata is too.
When checking for the href '/uploads/attachment/file/1/test_file.dat' in your tests you shouldn't be checking for a hardcoded '1' there you should check based on the record id number of the record you created. So "/uploads/attachment/file/#{question.id}/test_file.dat" (obviously question.id will depend on what objects you've created and your variable names, but that matches at least one of your tests I think.
Additionally after a quick look at your specs - anywhere you're doing expect(current_path).to eq ... is wrong. You should be doing expect(page).to have_current_path(...). The first way disables the retrying behavior of Capybara and will lead to flaky tests.
And finally - where you have expect(page).not_to have_content t('common.button.ready'), I'm guessing that should be expect(page).not_to have_button t('common.button.ready')
Here you could read how to config a database_cleaner with RSpec and Capybara - https://github.com/DatabaseCleaner/database_cleaner#rspec-with-capybara-example
I have a few tests in my suite that absolutely need an empty database to run.
Thing is, rails is automatically loading the fixtures for every test, and I can't seem to find a way to have it not load for this specific test.
I could drop the database before each test, but that's slow and requires all other tests that require fixtures to reload the fixtures afterwards.
Is there any way to have a certain test class (i.e. "NoFixturesTest") not load the fixtures (or unload them), without breaking all other tests?
Thanks!
EDIT:
Thanks to the answer below, I made this:
module DisableFixturesHelper
def self.included(base)
base.setup :setup_drop_db
# Have a unit test double-check that the fixtures are really gone
base.test "0 fixtures not loaded" do
assert_equal 0, Table1.count, "Table1 isn't empty!"
#etc
end
end
# Call to reset the db and therefore disable fixtures during a single unit test
def setup_drop_db
Rails.logger.info { "Dropping database." }
DatabaseCleaner.clean_with :truncation
end
end
And then in the TestCases that require an empty database (no fixtures), I add
include DisableFixturesHelper
The problem with this is that this drops the database once per test, AFTER the fixtures have already been loaded, so it's really slow. It's literally reloading the fixtures before the start of each unit test, just to drop them.
Still, it works.
do you know DatabaseCleaner ???
maybe it can help, you can also define what don't delete
something like:
DatabaseCleaner.strategy = :truncation, {:except => %w[widgets]}
here is an initializer that I always use:
RSpec.configure do |config|
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each, :js => true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
DatabaseCleaner.start
end
config.append_after(:each) do
DatabaseCleaner.clean
end
end
When testing my Models with RSpec I use the following code to clean my database:
config.before(:suite) do
begin
DatabaseCleaner.start
ensure
DatabaseCleaner.clean
end
end
config.after(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
All my Model tests pass, but when I test my controllers they appear to use the same dataset that my models operate on, making my tests error out.
I can get my Controller tests to pass by using this code instead:
config.before(:suite) do
DatabaseCleaner.strategy = :transaction
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
This however, causes my Model tests to fail. Any advice on how to either combine these blocks without breaking one set of tests or the other?
You didn't mention any specific versions of database_cleaner, nor your database. But the latest config for database_cleaner in RSpec uses an around filter
https://github.com/DatabaseCleaner/database_cleaner#rspec-example
The database is not being cleaned after each integration test. The value stays in the database.
Is there an option I should have to make this happen?
Thanks
I think https://github.com/bmabey/database_cleaner is what you need.
For anyone using before(:all) hooks, be aware that these hooks are executed before the transaction associated to the fixture is opened. This means that any data created by before(:all) hooks will not be rolled back by transactional fixtures. You can read more in the RSpec documentation.
I just wanted to mention this because I was bit by it and my initial instinct was to jump to Database Cleaner (which wound up not being needed and eventually not working).
How do I prepare test database(s) for Rails rspec tests without running rake spec?
My answer there might be of interest to you. it's a nice solution. For your case, you would probably need something like
config.after :each do
ActiveRecord::Base.subclasses.each(&:delete_all)
end
Look here for a tutorial: http://railscasts.com/episodes/257-request-specs-and-capybara
It describes Database Cleaner besides Rspec and Capybara
You want DatabaseCleaner, but you may find that the :truncation strategy is a bit too slow to run all the time. It's really only necessary for integration tests, so you can do this:
# spec/spec_helper.rb
require 'database_cleaner'
config.before(:suite) do
DatabaseCleaner.clean_with :truncation
DatabaseCleaner.strategy = :transaction
end
config.before(:each) do |group|
# The strategy needs to be set before we call DatabaseCleaner.start
case group.example.metadata[:type]
when :feature
DatabaseCleaner.strategy = :truncation
else
DatabaseCleaner.strategy = :transaction
end
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
# spec/features/your_feature_spec.rb
require 'spec_helper'
describe "An integration test", :type => :feature do
end
# spec/model/your_model_spec.rb
require 'spec_helper'
describe "A unit test" do
end
Obviously, this only applies if you're doing integration tests with RSpec directly vs. doing them with Cucumber.
There are two ways to accomplish this:
Configure transactional examples for each individual test.
Configure transactional examples for all the tests.
If you opt for option 1: At the top of the spec file, after:
require 'spec_helper'
Add:
RSpec.configure {|c| c.use_transactional_examples = true }
That will roll back the transactions after each example.
2.If you want to configure it globally, then, in the spec_helper.rb
RSpec.configure do |config|
...
config.use_transactional_examples = true # Add this
...
end
I'm running rails 3.0.3 and using rspec-rails 2.4.1 with a postgresql database. Whenever I run my RSpec tests, the data remains at the end. Does anyone know how to get rails or rspec to wipe the test environment's data between each use?
Please tell me if there's any further information that could make answering my question easier.
Thanks!Tristan
Install the database_cleaner gem and then add this to your spec_helper.rb.
Spec::Runner.configure do |config|
config.before(:suite) do
DatabaseCleaner.strategy = :transaction
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
end
Use transactional examples to rollback the data after every test run
RSpec.configure do |config|
config.use_transactional_examples = true
end
You don't need any extra gem in order to clean your test DB between runs. In your spec_helper.rb file, configure rspec as follows:
RSpec.configure do |c|
c.around(:each) do |example|
ActiveRecord::Base.connection.transaction do
example.run
raise ActiveRecord::Rollback
end
end
end
Another possibility, that I just put myself through, is using the wrong before block.
I accidentally set a before block as an all instead of an each:
before :all do
user = FactoryGirl.create(:user)
sign_in user
end
This caused the user to stick around in the database for the entire rspec run, which caused validation collisions.
Instead, the before should be an each so that everything is kept clean through the rspec run:
before :each do
user = FactoryGirl.create(:user)
sign_in user
end
If you've made this mistake, then you will probably need to manually clean up your test database before things go back to normal. The simplest way to do that is probably to truncate each of the tables (aside from schema_migrations).