I'm writing a gem for catching flaky tests. The idea is to dump database to locate randomized factory. I'm using system call in the .after hook:
config.after do |example|
system "pg_dump -U postgres -d #{db_name} > #{path}/#{status}.sql"
end
At this moment database contains a record - i've made sure of that by byebug.
However i bump into a problem - as i see, transaction is not closed, so if i try get record via psql - i get nothing, record doesn't exist.
Any ideas how to close transaction during spec? Or maybe i'm wrong somewhere deeply?
Yes, the matter is the transaction was not closed. Database cleaner wrap all rspec execution in begin-rollback. I found two ways to solve my problem:
Use active record connection to dump database content, but it's no so easy - you have to use COPY command to copy (unexpected, huh?) ALL tables to file (.csv for example) and then restore them. Ew, tedious.
Change database cleaner strategie from :transaction to :truncation:
config.before(:each, flaky: true) do
DatabaseCleaner.strategy = :truncation
end
That's the solution.
Related
In Rspec, I'm creating records, e.g. let!(:user) { create(:user) }.
I can access the new user in Rspec and in the main thread of the subject. Both of these return the user:
puts User.all.to_a
puts ActiveRecord::Base.connection.exec_query('select * from users')
However, I can now longer access the user in a new thread:
Thread.new do
puts User.all.to_a
puts ActiveRecord::Base.connection.exec_query('select * from users')
end
How do I fix this? Is this just an Rspec issue? I don't think I can reproduce it in Rails console.
You have probably configured RSpec to run its test within a database transaction
Quote from the RSpec Docs:
>
When you run rails generate rspec:install, the spec/rails_helper.rb file
includes the following configuration:
RSpec.configure do |config|
config.use_transactional_fixtures = true
end
The name of this setting is a bit misleading. What it really means in Rails
is "run every test method within a transaction." In the context of rspec-rails,
it means "run every example within a transaction."
The idea is to start each example with a clean database, create whatever data
is necessary for that example, and then remove that data by simply rolling back
the transaction at the end of the example.
You might want to disable this configuration when your application tests have the requirement to support multiple threads at the same time.
But that means that your test database will not be cleared from test data automatically anymore when a test is done. Instead, you will need to delete the test data manually or use another gem (like database_cleaner) to reset the database after running tests.
I've recently started testing my app using RSpec. Being the testing noob that i am, i did not install the full suite of tools that people normally use (namely FactoryGirl, also now known as FactoryBot. And DatabaseCleaner).
Anyway here's what happened. After getting my feet wet with RSpec, i started using FactoryBot so that my tests looks less convoluted. Note that :user is a Devise object.
# spec/models/first_test_spec.rb
FactoryBot.create(:user)
#Other Code
So i'm doing stuff with my first_test_spec.rb and i run it and everything is working perfectly fine. I then create my 2nd spec file
# spec/models/second_test_spec.rb
FactoryBot.create(:user)
#Other Code
Now here comes the problem. My tests are failing because FactoryBot.create(:user) is invalid, due to Devise requiring unique emails. Clearly this indicates that the data from my first_test_spec is persisting hence the error.
So i attempt to install DatabaseCleaner and hopefully clear my Test DB after each run. My rails_helper looks like this:
require 'spec_helper'
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
abort("The Rails environment is running in production mode!") if Rails.env.production?
require 'rspec/rails'
require 'support/factory_bot'
# Dir[Rails.root.join('spec/support/**/*.rb')].each { |f| require f }
ActiveRecord::Migration.maintain_test_schema!
RSpec.configure do |config|
config.fixture_path = "#{::Rails.root}/spec/fixtures"
config.use_transactional_fixtures = false
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each, js: true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
config.infer_spec_type_from_file_location!
config.filter_rails_from_backtrace!
end
I think i've got everything set up correctly, so i'm uncertain if the errors are still occurring due to my DatabaseCleaner being set up wrongly.
Anyway, running rspec still throws the same error, where second_test_spec cannot create the user object because the email already exists (a validation that comes with Devise itself).
I then proceed to run the following:
rake db:test:prepare
rspec spec/models/second_test_spec.rb
And it still throws the same error. So right now, i have no idea if my database is being cleanse after running rspec. But i'm quite certain that i have been unable to purge my test database.
So i guess i really have 2 questions/problems:
1) How do you purge your test database? (Googling reveals that rake db:test:prepare is the way)
2) Is my DatabaseCleaner setup correctly? If so, shouldn't it be purging the database?
UPDATE:
As suggested to me in the comments, using sequence for creating unique fields with FactoryBot is recommended. Obviously that made the problem go away because there would no longer be validation errors from Devise.
I then went on to test a couple of things.
rails c test
I ran this to check my test database and it was indeed empty. This indicates that DatabaseCleaner is working perfectly fine. What i fail to understand then, is why the need to sequence my email creation in FactoryBot? Or i suppose, i fail to understand how does RSpec "compile & run".
puts #user.email
So i wanted to print out the emails to look at the sequencing to see if i'm able to decipher the problem. Here's what happens:
running rspec spec/models/first_test_spec.rb
Tutor email yields the number 2.
running rspec spec/models/second_test_spec.rb
Tutor email yields the number 3.
running rspec
Tutor email yields the numbers 2 & 5.
So i'm not sure if there are "problems" with my test suite. Clearly my original problem has been fixed, and this is a separate topic altogether. But i figured if anyone would like to explain this mystery to anyone else who chances upon this thread may wish to do so.
Seeing your full spec files would help. If you are running the specs as you've written - creating a user outside of an example group or a before block - that will cause records to be written outside of the cleaning strategy scope and can cause data to remain across examples. Your DBcleaner config looks to be set up fine otherwise.
rake db:test:prepare is the correct way to clean out your test db but shouldn't need to be ran often unless you have migration changes. You can jump into a rails console within the test environment rails c test and look around to see if there are any records left behind.
As a side note you can flip config.use_transactional_fixtures from false to true and remove DBcleaner altogether from your app. Just make sure there is no residual data in your database before going this route.
I have an After hook in hooks.rb that deletes users created in the last scenario.
I started to notice that when tests run at a specific time of the day, this hook is being executed in the middle of a scenario.
There is a method that executes until a certain line and then the hook executes just before an assert command in that method, which fails because of it.
The tests are run from a batch file ("ruby file_name.rb").
Does anyone have an idea why this might happen or how to solve it?
Thanks!
Don't you run your tests from command line like following?
$ cucumber
I would suggest to use debugger gem. You could add debugger statement just before you think is failing and then use some debugger commands
https://github.com/cldwalker/debugger
Perhaps related to:
https://github.com/cucumber/cucumber/issues/52
Issue 52 is mostly fixed on the master, but I think there are a few remaining tests broken that need to be fixed before release.
Regardless of that, you might instead try using the database_cleaner gem for this purpose in general. We use a clean database before every scenario for testing to ensure we have discrete tests that cannot have false positive/negative results due to results of other tests. We use the following:
begin
# start off entire run with with a full truncation
DatabaseCleaner.clean_with :truncation
# continue with the :transaction strategy to be faster while running tests.
DatabaseCleaner.strategy = :transaction
rescue NameError
raise "You need to add database_cleaner to your Gemfile (in the :test group) if you wish to use it."
end
And we load our test seeds before each run:
Before do |scenario|
load Rails.root.join('db/seeds.rb')
end
Note that our seeds.rb checks which environment it is running in to keep it short. A big seeds file run in this manner would significantly increase test run times, so be careful.
I'm using Rspec to test my code against a legacy database (no migration, no schema etc). Is there a way to prevent the database from being dropped between iterations? Thanks.
config.before(:suite) do
DatabaseCleaner.strategy = nil
end
Try the https://github.com/bmabey/database_cleaner gem.
DatabaseCleaner.strategy = nil # will not do any db cleaning
I hope you can test on a local db before trying it on the real legacy db!
I am writing a scenario for signup form.
#abc
#selenium
Scenario:Non registered user signs up
Given I am on the sign-up page
When I fill in the following:
|first_name|Anidhya|
|last_name|Ahuja|
|email|anidhya#gmail.com|
|password|123456|
And I press "submit"
Then I should see "Registration complete"
I want to use database cleaner to roll back the test database after this scenario so that I can use this scenario again and again.
For that inside my env.rb file I wrote:
begin
require 'database_cleaner'
require 'database_cleaner/cucumber'
DatabaseCleaner.strategy = :transaction
Cucumber::Rails::World.use_transactional_fixtures = true
rescue NameError
raise "You need to add database_cleaner to your Gemfile (in the :test group) if you wish to use it."
end
Before('#abc') do
DatabaseCleaner.start
end
After('#abc') do
DatabaseCleaner.clean
end
Now when I run the scenario , the user gets saved in the database and the database cleaner fails. I dont see any error messages
Could you please clarify how to use database cleaner for only one scenario.I only want to use cleaner for this scenario.
Also could you please also provide the vital difference between using truncation and transaction.I think truncation clears the whole database but I dont want that.
Is there a better method of doing signup testing than this?
You can't run transactions with selenium because the test runs on two separate instances of the app AFAIK