Work is transitioning from Rails 3 to Rails 4. Everything seems to be running more or less smoothly on the development side, but testing results in a multitude of varying failures when all suites - units, functionals, and integration - are run in one go with rake:test.
What's interesting is that running each of these suites individually produces no failures. This strongly suggests that the database is not resetting for our tests in Rails 4 quite the way it was in Rails 3.
I've tried overriding the rake:test task to execute db:test:prepare prior to running each suite, but this apparently doesn't do what I think it does, or rather, what I want it to do, which is to work with a fresh set of data for each test and therefore succeed or fail independently of every other test - the way it (was or should have been) in our Rails 3 setup.
Why is this happening? How might I fix it?
(Note that the testing framework is vanilla Rails with fixtures and the like. The testing failures we're getting are usually foreign key errors or failures for data to change in expected ways that don't show up when the DB is reset before a testing suite.)
EDIT:
Here are the changes I'm making to my Rakefile. The idea is to get rake:test to perform as it would in Rails 3, with the exception of properly resetting the database between unit, functional, and integration suites.
# /Rakefile
task :test => [];
Rake::Task[:test].clear
task :test do
puts Rails.env
if Rails.env == "test"
puts "units"
# Rake::Task["db:drop"].execute - results in 'test database not configured' error
# Rake::Task["db:reset"].execute - results in 'relation 'table' does not exist' error
Rake::Task["db:create"].execute
Rake::Task["db:migrate"].execute
Rake::Task["test:units"].execute
puts "functionals"
Rake::Task["db:schema:load"].execute
Rake::Task["test:functionals"].execute
puts "integration"
Rake::Task["db:schema:load"].execute
Rake::Task["test:integration"].execute
end
end
EDIT 2:
Rails version 4.1.8
Here is my test_helper. I've omitted helper methods that don't work with our data since I'm probably not allowed to share them and they're irrelevant to how the testing database is loaded.
Note also that we have a custom schema.rb from which the testing data are generated, as well, since trying to create them from migrations does not support some of the custom functionality/behavior we want in the end product, like materialized views. Redacted stuff will be enclosed in <<>>.
#test_helper.rb
ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
include ActionDispatch::TestProcess
class ActiveSupport::TestCase
# Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
#
# Note: You'll currently still have to declare fixtures explicitly in integration tests
# -- they do not yet inherit this setting
set_fixture_class <<cached_object>>: ResultsDb::<<DesiredObjectClass>>Cache
fixtures :all
# Add more helper methods to be used by all tests here...
def reload_contig(contig_sym)
contig = contigs(contig_sym)
src = contig.src
new_tig = Contig.new(
org: contig.org,
version: contig.version,
size: contig.size,
sha1: contig.sha1,
active: contig.active,
src_id: src.id,
src_type: contig.src_type)
new_tig.id = contig.id
contig.destroy
unless new_tig.save
raise new_tig.errors.full_messages.inspect
end
src.contigs << new_tig
src.save
new_tig
end
def reload_coord(coord_sym)
coord = coords(coord_sym)
new_coord = Coord.new(
:mapped_id => coord.mapped_id,
:mapped_type => coord.mapped_type,
:contig_id => coord.contig_id,
:contig_type => coord.contig_type,
:start => coord.start,
:stop => coord.stop,
:fwd_strand => coord.fwd_strand
)
new_coord.id = coord.id
coord.destroy
new_coord.save!
end
end
You might have a look at the DatabaseCleaner gem that is specifically designed to assist you with database consistency and cleanliness during testing in Rails.
I've only run into problems when my code bypasses ActiveRecord and manipulates the data directly. In all other cases this gem has worked very well for me for testing.
Assuming you are currently on Rails 4.0, your current approach is about the best you can do if you can't add the database_cleaner gem. (I would suggest that you prefer db:schema:load to db:migrate throughout—there's no reason ever to run through all the migrations from the start in the test environment, unless you've done something unusual with them.) But once you get to Rails 4.1 or higher, you've got a better solution available.
Starting with Rails 4.1, management of the test database schema was automated, and rake db:test:prepare was deprecated. If you regenerate your test_helper.rb at this point, everything should work magically.
Have you tried adding either of these in your overriding of rake:test?
rake db:reset db:migrate
or
rake db:drop db:create db:migrate
The question seems similar to this one. I know I've always built my own rake db:recreate that does this sort of thing for test data.
Related
My legacy code is using DatabaseCleaner which wipes out database after each test being done. However it also wipes out loaded fixtures data and causes some tests which are using fixtures to fail, something like:
ActiveRecord::RecordNotFound Exception: Couldn't find Country with 'id'=593363170
To make them work together, how can I rewrite all fixtures back to database during test? (I mean not rake tasks)
Solution:
Set self.use_transactional_fixtures = false
at the beginning of the file before which you want to reload fixtures.
Or use use_transactional_tests in Rails 5.
Do not set it in spec_helper as it applies globally so conflicts with DatabaseCleaner.
I have integration tests that are intermittently failing, and always with ActiveRecord::RecordNotFound errors. The error takes place inside the controller where a find call takes place given an ID from a fixture. It takes place in multiple controllers, though. I never see this behaviour while navigating the site, but I'd say the tests fail about 30-50% of the time. Running the test again after a failure seems to fix the problem.
If I load the fixtures manually into the development database, the ID that doesn't seem to be found is indeed present inside the tables.
I haven't been able to find much info on people having the same problem... any ideas?
UPDATE: Here are the contents of test_helper.rb
ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require 'capybara/rails'
class ActiveSupport::TestCase
# Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
#
# Note: You'll currently still have to declare fixtures explicitly in integration tests
# -- they do not yet inherit this setting
fixtures :all
# Add more helper methods to be used by all tests here...
end
# Transactional fixtures do not work with Selenium tests, because Capybara
# uses a separate server thread, which the transactions would be hidden
# from. We hence use DatabaseCleaner to truncate our test database.
DatabaseCleaner.strategy = :truncation
class ActionDispatch::IntegrationTest
# Make the Capybara DSL available in all integration tests
include Capybara::DSL
# Make the Capybara Email DSL available in all integration tests
include Capybara::Email::DSL
# Stop ActiveRecord from wrapping tests in transactions
self.use_transactional_fixtures = false
# Switch to selenium as the default driver for JS support
Capybara.default_driver = :selenium
# Only click on visible links!
Capybara.ignore_hidden_elements = true
teardown do
DatabaseCleaner.clean # Truncate the database
Capybara.reset_sessions! # Forget the (simulated) browser state
Capybara.use_default_driver # Revert Capybara.current_driver to Capybara.default_driver
end
end
UPDATE Here are the results of running the same test 5 times in a row. I ran the tests, waited until they finished, and then immediately ran them again with the command rails test:integration. NOTE: The E and F that are consistent throughout the tests are actually test errors -- I'm working on fixing those. For example, the second test run-through was "correct", but the first showed the spurious error.
..E......E..........F.
.........E..........F.
..E......E..........F.
..E......E..........F.
..E....E.E..........F.
The errors do occur across two separate tables -- they're not trying to find the same record. But it does appear to be only a subset of the tests that have this issue...
UPDATE Here's what the actual error looks like in the test results:
1) Error:
test_browsing_user_snops(BrowseStoriesTest):
ActiveRecord::RecordNotFound: Couldn't find User with id=980190962
/home/myuser/.rvm/gems/ruby-1.9.3-p286/gems/activerecord-3.2.9/lib/active_record/relation/finder_methods.rb:341:in `find_one'
/home/myuser/.rvm/gems/ruby-1.9.3-p286/gems/activerecord-3.2.9/lib/active_record/relation/finder_methods.rb:312:in `find_with_ids'
/home/myuser/.rvm/gems/ruby-1.9.3-p286/gems/activerecord-3.2.9/lib/active_record/relation/finder_methods.rb:107:in `find'
/home/myuser/.rvm/gems/ruby-1.9.3-p286/gems/activerecord-3.2.9/lib/active_record/querying.rb:5:in `find'
/home/myuser/Projects/myproject/app/controllers/users_controller.rb:18:in `show'
...
Notice that the error appears when the controller tries to find a record. The relevant line is actually #user = User.find(params[:id]). It occurs with other models also, not just the users controller and not just the User model.
I'm concerned that there may be a delay during the truncation of data and the speed in which Capybara is driving the web browser which sometimes may result in identity column values starting at an unexpected value (i.e., 7 for record #1 - when you expect 1, because that identity generator has not yet been reset). I have no evidence that this is the case but its my best guess.
Take at look at item #3 at this URL. This is a pretty straight-forward hack, set use_transactional_fixtures to true and monkeypatch ActiveRecord by pasting that code into your test_helper.rb. This could help in eliminating any intermittent disk IO problems that could be a potential problem.
Another thing you can try is restricting your SQLite test database by setting the filename for that database to :memory: in your database.yml file. This should accomplish the same thing as above - eliminate spurious disk IO that may be causing these intermittent issues.
Today, I use the factory_girl instead of the rails fixtures, but i get a problem:
After I run the command "spec spec" done, the data resets to the fixtures, who can tell me the answer?
thank you!
If you intend to use both factories and fixtures in your project and not running them through the rake tasks: eg rake spec, you will need to make sure that you are doing removal of the values from the db by hand. its likely what you are doing is just grabbing an old record in the database and you think the data is being reset. You can verify this by using puts inside your spec to trace the number of records in the db.
puts MyRecord.count
You can clear values in an after or before block.
before(:each) do
Factory(:my_model)
end
after(:each) do
MyModel.delete_all
end
if you intend to use this model or factory in other spec files, you can add these to global before and after blocks in the spec helper.
rake test:units fails in my current application, because the needed data of the fixtures is missing.
If I'm loading the fixtures manually via rake db:fixtures:load RAILS_ENV=test the unit tests are working, but rake purges the test database.
My test_helper includes fixtures :all and my tests are inheriting from it - but the fixtures are simply not loading.
I'm kind of clueless at the moment and could really need some help!
I've tried a lot and I think it has to do with some environment settings or plugins used in this project. Does anyone know where to read about which files are loaded for the testing environment?
Put the call to fixtures :all in your test class, not the super class (test_helper). My guess is that initialization of the super class isn't working the way you're expecting and that fixtures :all isn't be called. Perhaps try putting the call in the initialize method of test_helper.
My test/test_helper.rb looks like this:
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
class ActiveSupport::TestCase
# Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
fixtures :all
# Add more helper methods to be used by all tests here...
end
I finally found the problem, although the solutions is kind of hacky.
One plugin is relying that there is some data in the database, at least one row. So what happened was:
rake loads database schema
rake tries to load environment
environment includes plugin
plugin loading fails because of missing at least one row
no fixtures are loaded
The hacky solution is: put the needed data directly into schema and not into a fixtures, because it's loaded to late.
I'll search for a more convenient solution and will update this answer if I found one.
Another approach is to write your own custom rake task for testing.
For example:
task :test_units do
RAILS_ENV = 'test' # Force the environment to test
puts "Recreate the test database"
Rake::Task['db:test:prepare'].invoke
puts "Seed the database with fixtures"
Rake::Task['db:fixtures:load'].invoke
puts "Executing Unit Tests"
Rake::Task['test:units'].prerequisites.clear
Rake::Task['test:units'].invoke
end
My problem is forgot to put "require 'test_helper'" at the head.
eg.
require 'test_helper'
class AdminUserTest < ActiveSupport::TestCase
# test "the truth" do
# assert true
# end
end
I had the same problem. Or rather, the problem was that my fixtures were not current with the database schema. Instead of throwing an exception, rails just used the test database as a fallback solution. Most iffy.
I have fixtures with initial data that needs to reside in my database (countries, regions, carriers, etc.). I have a task rake db:seed that will seed a database.
namespace :db do
desc "Load seed fixtures (from db/fixtures) into the current environment's database."
task :seed => :environment do
require 'active_record/fixtures'
Dir.glob(RAILS_ROOT + '/db/fixtures/yamls/*.yml').each do |file|
Fixtures.create_fixtures('db/fixtures/yamls', File.basename(file, '.*'))
end
end
end
I am a bit worried because this task wipes my database clean and loads the initial data. The fact that this is even possible to do more than once on production scares the crap out of me. Is this normal and do I just have to be cautious? Or do people usually protect a task like this in some way?
Seeding data with fixtures is an extremely bad idea.
Fixtures are not validated and since most Rails developers don't use database constraints this means you can easily get invalid or incomplete data inserted into your production database.
Fixtures also set strange primary key ids by default, which is not necessarily a problem but is annoying to work with.
There are a lot of solutions for this. My personal favorite is a rake task that runs a Ruby script that simply uses ActiveRecord to insert records. This is what Rails 3 will do with db:seed, but you can easily write this yourself.
I complement this with a method I add to ActiveRecord::Base called create_or_update. Using this I can run the seed script multiple times, updating old records instead of throwing an exception.
I wrote an article about these techniques a while back called Loading seed data.
For the first part of your question, yes I'd just put some precaution for running a task like this in production. I put a protection like this in my bootstrapping/seeding task:
task :exit_or_continue_in_production? do
if Rails.env.production?
puts "!!!WARNING!!! This task will DESTROY " +
"your production database and RESET all " +
"application settings"
puts "Continue? y/n"
continue = STDIN.gets.chomp
unless continue == 'y'
puts "Exiting..."
exit!
end
end
end
I have created this gist for some context.
For the second part of the question -- usually you really want two things: a) very easily seeding the database and setting up the application for development, and b) bootstrapping the application on production server (like: inserting admin user, creating folders application depends on, etc).
I'd use fixtures for seeding in development -- everyone from the team then sees the same data in the app and what's in app is consistent with what's in tests. (Usually I wrap rake app:bootstrap, rake app:seed rake gems:install, etc into rake app:install so everyone can work on the app by just cloning the repo and running this one task.)
I'd however never use fixtures for seeding/bootstrapping on production server. Rails' db/seed.rb is really fine for this task, but you can of course put the same logic in your own rake app:seed task, like others pointed out.
Rails 3 will solve this for you using a seed.rb file.
http://github.com/brynary/rails/commit/4932f7b38f72104819022abca0c952ba6f9888cb
We've built up a bunch of best practices that we use for seeding data. We rely heavily on seeding, and we have some unique requirements since we need to seed multi-tenant systems. Here's some best practices we've used:
Fixtures aren't the best solution, but you still should store your seed data in something other than Ruby. Ruby code for storing seed data tends to get repetitive, and storing data in a parseable file means you can write generic code to handle your seeds in a consistent fashion.
If you're going to potentially update seeds, use a marker column named something like code to match your seeds file to your actual data. Never rely on ids being consistent between environments.
Think about how you want to handle updating existing seed data. Is there any potential that users have modified this data? If so, should you maintain the user's information rather than overriding it with seed data?
If you're interested in some of the ways we do seeding, we've packaged them into a gem called SeedOMatic.
How about just deleting the task off your production server once you have seeded the database?
I just had an interesting idea...
what if you created \db\seeds\ and added migration-style files:
file: 200907301234_add_us_states.rb
class AddUsStates < ActiveRecord::Seeds
def up
add_to(:states, [
{:name => 'Wisconsin', :abbreviation => 'WI', :flower => 'someflower'},
{:name => 'Louisiana', :abbreviation => 'LA', :flower => 'cypress tree'}
]
end
end
def down
remove_from(:states).based_on(:name).with_values('Wisconsin', 'Louisiana', ...)
end
end
alternately:
def up
State.create!( :name => ... )
end
This would allow you to run migrations and seeds in an order that would allow them to coexist more peaceably.
thoughts?