I've recently started testing my app using RSpec. Being the testing noob that i am, i did not install the full suite of tools that people normally use (namely FactoryGirl, also now known as FactoryBot. And DatabaseCleaner).
Anyway here's what happened. After getting my feet wet with RSpec, i started using FactoryBot so that my tests looks less convoluted. Note that :user is a Devise object.
# spec/models/first_test_spec.rb
FactoryBot.create(:user)
#Other Code
So i'm doing stuff with my first_test_spec.rb and i run it and everything is working perfectly fine. I then create my 2nd spec file
# spec/models/second_test_spec.rb
FactoryBot.create(:user)
#Other Code
Now here comes the problem. My tests are failing because FactoryBot.create(:user) is invalid, due to Devise requiring unique emails. Clearly this indicates that the data from my first_test_spec is persisting hence the error.
So i attempt to install DatabaseCleaner and hopefully clear my Test DB after each run. My rails_helper looks like this:
require 'spec_helper'
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
abort("The Rails environment is running in production mode!") if Rails.env.production?
require 'rspec/rails'
require 'support/factory_bot'
# Dir[Rails.root.join('spec/support/**/*.rb')].each { |f| require f }
ActiveRecord::Migration.maintain_test_schema!
RSpec.configure do |config|
config.fixture_path = "#{::Rails.root}/spec/fixtures"
config.use_transactional_fixtures = false
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each, js: true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
config.infer_spec_type_from_file_location!
config.filter_rails_from_backtrace!
end
I think i've got everything set up correctly, so i'm uncertain if the errors are still occurring due to my DatabaseCleaner being set up wrongly.
Anyway, running rspec still throws the same error, where second_test_spec cannot create the user object because the email already exists (a validation that comes with Devise itself).
I then proceed to run the following:
rake db:test:prepare
rspec spec/models/second_test_spec.rb
And it still throws the same error. So right now, i have no idea if my database is being cleanse after running rspec. But i'm quite certain that i have been unable to purge my test database.
So i guess i really have 2 questions/problems:
1) How do you purge your test database? (Googling reveals that rake db:test:prepare is the way)
2) Is my DatabaseCleaner setup correctly? If so, shouldn't it be purging the database?
UPDATE:
As suggested to me in the comments, using sequence for creating unique fields with FactoryBot is recommended. Obviously that made the problem go away because there would no longer be validation errors from Devise.
I then went on to test a couple of things.
rails c test
I ran this to check my test database and it was indeed empty. This indicates that DatabaseCleaner is working perfectly fine. What i fail to understand then, is why the need to sequence my email creation in FactoryBot? Or i suppose, i fail to understand how does RSpec "compile & run".
puts #user.email
So i wanted to print out the emails to look at the sequencing to see if i'm able to decipher the problem. Here's what happens:
running rspec spec/models/first_test_spec.rb
Tutor email yields the number 2.
running rspec spec/models/second_test_spec.rb
Tutor email yields the number 3.
running rspec
Tutor email yields the numbers 2 & 5.
So i'm not sure if there are "problems" with my test suite. Clearly my original problem has been fixed, and this is a separate topic altogether. But i figured if anyone would like to explain this mystery to anyone else who chances upon this thread may wish to do so.
Seeing your full spec files would help. If you are running the specs as you've written - creating a user outside of an example group or a before block - that will cause records to be written outside of the cleaning strategy scope and can cause data to remain across examples. Your DBcleaner config looks to be set up fine otherwise.
rake db:test:prepare is the correct way to clean out your test db but shouldn't need to be ran often unless you have migration changes. You can jump into a rails console within the test environment rails c test and look around to see if there are any records left behind.
As a side note you can flip config.use_transactional_fixtures from false to true and remove DBcleaner altogether from your app. Just make sure there is no residual data in your database before going this route.
Related
I'm writing a feature test using Capybara and RSpec where the app is remote (i.e. not started by Capybara). The test and the app share the same database. So I assumed that any change the test makes to the database would be available to the remote app as well.
But when I use Selenium to fill in forms with data from the records I created in my test steps, I get "No such record" errors.
My RSpec configuration includes this line:
RSpec.configure do |c|
c.use_transactional_fixtures = false
end
I assumed this would allow the data to be available for Selenium. But that's not what is happening.
My Capybara configuration is:
Capybara.javascript_driver = :headless_chrome
Capybara.app_host = 'http://selenium:5000'
Capybara.run_server = false
Both the test and the app under test use the same database at DATABASE_URL=postgres://postgres#db:5432/test_db.
Can anyone help me out?
Figure out that DatabaseCleaner was adding the transaction block.
DatabaseCleaner.strategy = :transaction
I have integration tests that are intermittently failing, and always with ActiveRecord::RecordNotFound errors. The error takes place inside the controller where a find call takes place given an ID from a fixture. It takes place in multiple controllers, though. I never see this behaviour while navigating the site, but I'd say the tests fail about 30-50% of the time. Running the test again after a failure seems to fix the problem.
If I load the fixtures manually into the development database, the ID that doesn't seem to be found is indeed present inside the tables.
I haven't been able to find much info on people having the same problem... any ideas?
UPDATE: Here are the contents of test_helper.rb
ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require 'capybara/rails'
class ActiveSupport::TestCase
# Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
#
# Note: You'll currently still have to declare fixtures explicitly in integration tests
# -- they do not yet inherit this setting
fixtures :all
# Add more helper methods to be used by all tests here...
end
# Transactional fixtures do not work with Selenium tests, because Capybara
# uses a separate server thread, which the transactions would be hidden
# from. We hence use DatabaseCleaner to truncate our test database.
DatabaseCleaner.strategy = :truncation
class ActionDispatch::IntegrationTest
# Make the Capybara DSL available in all integration tests
include Capybara::DSL
# Make the Capybara Email DSL available in all integration tests
include Capybara::Email::DSL
# Stop ActiveRecord from wrapping tests in transactions
self.use_transactional_fixtures = false
# Switch to selenium as the default driver for JS support
Capybara.default_driver = :selenium
# Only click on visible links!
Capybara.ignore_hidden_elements = true
teardown do
DatabaseCleaner.clean # Truncate the database
Capybara.reset_sessions! # Forget the (simulated) browser state
Capybara.use_default_driver # Revert Capybara.current_driver to Capybara.default_driver
end
end
UPDATE Here are the results of running the same test 5 times in a row. I ran the tests, waited until they finished, and then immediately ran them again with the command rails test:integration. NOTE: The E and F that are consistent throughout the tests are actually test errors -- I'm working on fixing those. For example, the second test run-through was "correct", but the first showed the spurious error.
..E......E..........F.
.........E..........F.
..E......E..........F.
..E......E..........F.
..E....E.E..........F.
The errors do occur across two separate tables -- they're not trying to find the same record. But it does appear to be only a subset of the tests that have this issue...
UPDATE Here's what the actual error looks like in the test results:
1) Error:
test_browsing_user_snops(BrowseStoriesTest):
ActiveRecord::RecordNotFound: Couldn't find User with id=980190962
/home/myuser/.rvm/gems/ruby-1.9.3-p286/gems/activerecord-3.2.9/lib/active_record/relation/finder_methods.rb:341:in `find_one'
/home/myuser/.rvm/gems/ruby-1.9.3-p286/gems/activerecord-3.2.9/lib/active_record/relation/finder_methods.rb:312:in `find_with_ids'
/home/myuser/.rvm/gems/ruby-1.9.3-p286/gems/activerecord-3.2.9/lib/active_record/relation/finder_methods.rb:107:in `find'
/home/myuser/.rvm/gems/ruby-1.9.3-p286/gems/activerecord-3.2.9/lib/active_record/querying.rb:5:in `find'
/home/myuser/Projects/myproject/app/controllers/users_controller.rb:18:in `show'
...
Notice that the error appears when the controller tries to find a record. The relevant line is actually #user = User.find(params[:id]). It occurs with other models also, not just the users controller and not just the User model.
I'm concerned that there may be a delay during the truncation of data and the speed in which Capybara is driving the web browser which sometimes may result in identity column values starting at an unexpected value (i.e., 7 for record #1 - when you expect 1, because that identity generator has not yet been reset). I have no evidence that this is the case but its my best guess.
Take at look at item #3 at this URL. This is a pretty straight-forward hack, set use_transactional_fixtures to true and monkeypatch ActiveRecord by pasting that code into your test_helper.rb. This could help in eliminating any intermittent disk IO problems that could be a potential problem.
Another thing you can try is restricting your SQLite test database by setting the filename for that database to :memory: in your database.yml file. This should accomplish the same thing as above - eliminate spurious disk IO that may be causing these intermittent issues.
I'm trying to get database_cleaner working with my test suite and I can't find anything online about this error:
ERROR Couldn't find User with id=1
The tests run fine until I add database_cleaner. This is how I'm using it in my test_helper.rb file:
# Database cleaner.
DatabaseCleaner.strategy = :truncation
class MiniTest::Spec
before :each do
DatabaseCleaner.clean
end
end
It also does not work if my strategy is :transaction.
According to the docs, the call to DatabaseCleaner.clean should be in an after hook, and DatabaseCleaner.start in a before hook (the start call may be unnecessary with the truncation strategy.
What may well be happening here is that database cleaner is clearing out any data you've setup in other before hooks before your actual test runs. It will depend which order the before hooks run in.
I trying to get a backtrace with rspec but I can't get it working for some reason.
This is the test file:
require 'spec_helper'
describe ActivityReport do
it "should create a new instance given valid attributes" do
activity = Factory(:activity_report)
end
This is the command I run:
rspec --backtrace spec/models/activity_report_spec.rb
And this is what I get:
No examples matched {:focus=>true}. Running all.
ActivityReport
should create a new instance given valid attributes (FAILED - 1)
Failures:
1) ActivityReport should create a new instance given valid attributes
Failure/Error: Unable to find matching line from backtrace
SystemStackError:
stack level too deep
# /Users/pbartels/.rvm/gems/ruby-1.9.2-p290#brothelking/gems/activerecord-3.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:206
Finished in 40.76 seconds
1 example, 1 failure
Failed examples:
rspec ./spec/models/activity_report_spec.rb:16 # ActivityReport should create a new instance given valid attributes
My .rspec:
--format nested
--color
--drb
--backtrace
And my RSpec part in spec_helper.rb:
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
require 'factory_girl'
require 'database_cleaner'
require 'active_record/fixtures'
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.start
Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}
RSpec.configure do |config|
config.mock_with :rspec
config.use_transactional_fixtures = true
config.treat_symbols_as_metadata_keys_with_true_values = true
config.filter_run :focus => true
config.run_all_when_everything_filtered = true
config.backtrace_clean_patterns = [
/\/lib\d*\/ruby\//,
/bin\//,
#/gems/,
/spec\/spec_helper\.rb/,
/lib\/rspec\/(core|expectations|matchers|mocks)/
]
end
I tried it with and without "backtrace_clean_patterns".
Anyone knows what's wrong here?
There's no backtrace because it isn't a given line of code that's failing, it's actually the structure of your code. The ruby interpreter is literally running out of room to store further method calls on the stack.
Stack level too deep typically means that you've got recently added/modified code that is calling itself, and going into a infinitely recursive black hole. Look at the code that you've recently added (including tests) for the clue. See if you're calling a method from within itself.
For example, this code will cause stack overflow:
def recursive_method
recursive_method
end
You've probably got a method call or field that shares the name of a method, and when you reference it, it's going into this infinite loop/recursive call.
If that doesn't point you in the right direction, you may be forced to incrementally revert you recent code changes until the problem goes away. When you get back to a state in your code where the problem goes away, you'll know that the issue has something to do with that change, though the specifics may not be immediately clear.
The other option, if you're really not getting anywhere with that (or it's not feasible to revert changes in an intelligent way), is to start adding debug lines to your code where you suspect the problem might be. Somehow you'll need to get your app to write to a log file or something so you can figure out what it is that it's doing right before it dies.
For anyone arriving here looking for how to run rspec with trace, it's
rspec --backtrace
I have 30 specs in my foo_controller_spec.rb and when I run the entire file using spec, I get 4 failures and 2 pending. When I run the 4 failing specs individually, 3 of them still fail, but one of them passes. At first I thought it was a database issue, that the data wasn't being cleaned out properly between runs. So I installed database_cleaner (http://github.com/bmabey/database_cleaner) and added this code to my spec_helper:
config.before(:suite) do
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.start
Sham.reset
login
end
config.after(:each) do
DatabaseCleaner.clean
end
Now when I run my specs I can see that the tables are truncated between each spec so I know it's working, but the spec still fails when run with all the other specs but passes when run alone. What am I missing here?
I had exactly the same issue and was going out of my mind!
I tracked the issue like this (with Textmate):
select your files in project drawer (multiple files) and then go to "Bundles > Rspec > Run examples in selected files/directories".
The trick is to find which file is causing interference with others.
For me it was changing of I18n.locale in one file which caused the locale to be changed for all other examples!
I lost a few hours going nuts with this...
Just in case it helps others: I had a similar problem and found out I had
stray Rspec.configures that were overriding for all tests down the line
WebMocks enabled in one test that seem to cascade to following on tests until i I included in the spec_helper so connections were on by default
RSpec.configure do |config|
config.before(:suite) do
WebMock.allow_net_connect!
I Had a similar issue today that took quite a bit of time to resolve.
This fails:
module SomeModule
describe SomeController do
it 'does something'
end
end
This passes:
describe SomeModule::SomeController do
it 'does something'
end
The issue has to deal with scoping.
The command rspec --bisect, or --order rand:21237 --bisect if you see an error regarding ordering, is a good first step towards debugging. Add --bisect=verbose for more information.
Link to RSpec GH issue