We have a ton of non-user data: 500 product types that haven't changed in 20 years, 200GB of geospatial data that never changes (Queens is always 40.73N/73.82W)... etc. It's the same in Development and Production. It should be the same in Test, but the during testing Rails empties all of the test tables, and it takes a ton of time to re-populate during testing.
What is the Rails way to partition this non-user data so that it doesn't have to be repopulated in Test?
The documentation for this is found in the Fixtures class. (Search for "Transactional fixtures" on that page.)
The example they give starts out like this:
class FooTest < ActiveSupport::TestCase
self.use_transactional_fixtures = true
self.use_instantiated_fixtures = false
...
One of the projects I work on uses a test database with zero fixtures, so we just define this globally in test_helper.rb.
class ActiveSupport::TestCase
self.use_transactional_fixtures = true
self.use_instantiated_fixtures = false
end
jdl shows you the settings to use for enabling and disabling transactional_fixtures, you should be able to set:
# Use self in ActiveSupport::TestCase if you don't have a config block
# in test/test_helper.rb
#
config.use_transactional_fixtures = false
And stop Rails from trying to load fixtures before each run of your tests. The downside is you can't assume all your fixtures are loaded into the DB.
Real Answer
You have too much data in your fixtures. The Railsy thing to do it load only the necessary data in test fixtures; You do not need 200G of geospatial data (what freaking dataset is that anyway? that sounds way too big), and you probably don't need all 500 products.
Tests are for your code, so only include the few geospatial points you need for a test, or only include a few products with unique properties. Keep the DB light enough to be loaded quickly.
Your test fixtures should be separate from you application bootstrapping data, take advantage of that.
Real Real Answer
Don't touch the database in your tests (or, touch it very little). The database is slow and fixtures are difficult or impossible to maintain well. Instead, try using a stubbing framework like Mocha or flexmock. It's faster, and makes your tests read in a clearer manner. Tests are for code, you can stub the database out of the situation and trust that ActiveRecord works (because...it's tested too! :-).
If you do choose to stick with fixtures, I recommend using factory_girl instead of the built in Rails fixtures. You'll have a much better starting point.
Related
In rails rspec, I am writing test cases that do this:
after(:each) do
DatabaseCleaner.clean_with(:truncation)
end
I need the database to be cleaned after each test is run. But will this affect the performance of my tests and make them slower?
The "truncation" method has an impact based on the number of tables, because it will execute one query per table to truncate. You can use the "transaction" method instead, which will do a transaction-based rollback instead, which will drastically decrease the amount of resources used to roll back to a clean database state.
If you use ActiveRecord and want to have a clear picture of what happens, you can do something like this in your test:
before do
ActiveRecord::Base.logger = Logger.new(STDOUT)
end
It will have impact on performance depending on the number of tests on which the after hook will be run.
Also, I think it is best to use before instead of after. Depending on the test case, the state of the database is a precondition; the test case requires that precondition. And you should ensure that precondition with the before hook instead of relying on after hooks defined somewhere else.
I've been in Factory land for the past few years and have decided to come back to fixtures but am running into a problem.
In my test_helper.rb I have
class ActiveSupport::TestCase
fixtures :all
end
Then in an individual test case I might be doing something like users(:one) however the data for users(:two) and the data for other tables that I am not calling on seems to be present in the test database.
So, is this the expected behavior? I have a hard to believing it is, seems strange from an isolated testing perspective.
This happens because you load all the fixtures at once from fixtures :all statement. A rule of thumb in tests is to load only the required data for a given test (not all). This also could slow down your test running time.
If you want to load only the selected fixtures you could do
fixtures :<fixture name>
Ex:
fixtures :users
Read more about fixtures.
One more thing, do you have a particular reason to comeback to fixtures. Factories are the way to go; they allow you do organize your test data cleanly.
Read more about factories.
So far I use rspec and factories only, not fixtures or seed data. But now I need a (static) Coupon table with a dozen or so records of config data to be available to my app when running my integration tests (rspec + capybara, not cuke).
My rspec tests have a FactoryGirl factory that creates a Store.
In my app, when a Store is initialized by init_store(coupon_code), a coupon code is passed in that turns on/off various features (by looking up the settings for that coupon in Coupon table).
So, when I create a Store factory, I also need to specify (via trait?) a coupon code. Then, down in my model, the method that initializes a store will use the coupon code to lookup a bunch of settings in the Coupon table. (Each coupon has about 20 attributes that turn on/off various Store features.)
My coupons.yml file contains the actual coupon table data used by my app. Now I need to make them available to the app when tests are being run. (Don't care about 'brittle', this table doesn't change.)
Having not used fixtures OR seed data with rspec, I'm not sure how/where to put my coupons.yml file and how to make the coupons 'load' into the test Coupon table so they are available to my app while tests are running.
Do I pre-populate seed data with coupons.yml? If so, how can I have that get loaded into my test database each time my tests run?
Or, should I use fixtures?
Either way, where should coupons.yml go and how do I load it (in the Store factories? in one of the specs? In specs_helper?)
When I search for stuff like "using fixtures and factories" or "using seed data with rspec" all I find is the advice "use factories" and yeah, I get that. But for this particular situation I need both, so any help would be appreciated.
I do something like this, and I use seeds rather than fixtures (though I'm sure you could use fixtures too).
In spec_helper.rb
RSpec.configure do |config|
...
config.before :suite do
DatabaseCleaner.strategy = :truncation, {:except => %w[table1 table2]}
DatabaseCleaner.clean_with :truncation
Rails.application.load_seed
end
...
end
So the DatabaseCleaner.strategy line is telling database_cleaner to clean everything except the array of tables you pass it.
After setting the strategy, you simply call Rails.application.load_seed to load the seed data.
Hope that helps, let me know if you have any questions.
I've got this issue with database locking when I'm testing some threading features I've got in my application--the database locks on one thread and then all the other threads deadlock on that. As there's no explicit transaction in my code I can't just guard and release manually.
I read somewhere that fixtures in TestCase leverage implicit (implicit to MY code) transactions to clean out the database between tests, but I can't find anywhere if this is true for Factory Girl as well.
Does anyone know if it is true and if so, is there a way to turn it off for specific tests, but not all of the tests?
thanks in advance!
FactoryGirl just takes your model, sets the attributes, and calls #save on that model. I think you just need to change a setting. With RSpec, you should have a line in your spec_helper.rb file:
config.use_transactional_fixtures = true
This config is sent to the Rails testing config. This is then used in the #setup_fixtures method.
The documentation on Rails transactional fixtures
HTH
I have a good half-dozen fixtures in my Rails test suite - some quite large - that represent static look-up tables in the database. Since their values theoretically don't change, I'd like to save the time of wiping and reloading the tables between each test. Is there a trick/plug-in/gem/hack to demarcate these fixtures as only needing to be loaded once before all the tests run and not wiped between each?
You can desactivate rollback transaction in all your test.
Or you can add this fixtures before all of your test. Before the begin of transaction.