database_cleaner not clearing some of my tests (rspec, capybara, selenium) - ruby-on-rails

database_cleaner works for the most part but on some things where i am expecting something like user_id to clear after each test it doesn't. so, the user_id will increment throughout instead of clearing and the user id is predictable as 1, 2, 3 or however many that are created for the test. i can just call the id instead of hardcoding the expected result but later on i really need it to clear that stuff in more complex examples. this is the easiest for showing. any help would be greatly appreciated.
FROM SPEC_HELPER.RB:
RSpec.configure do |config|
config.mock_with :rspec
config.include FactoryGirl::Syntax::Methods
config.include(Capybara, :type => :integration)
config.include Devise::TestHelpers, :type => :controller
config.use_transactional_fixtures = false
config.before(:each) do
I18n.default_locale = :en
I18n.locale = :en
DatabaseCleaner.start
ResqueSpec.reset!
ActionMailer::Base.deliveries.clear
end
config.after(:each) do
DatabaseCleaner.clean
end
config.after(:all) do
TestCleaner.clean
end
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
DatabaseCleaner.strategy = :transaction
Role.reset_cache!
end
config.after(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
FROM MY TEST:
it "should return one provider" do
get :index
response.body.gsub(/\s+/, "").should == {
:experts => [{
:availability => false,
:name => "#{#provider.user.first_name}#{#provider.user.last_name}",
:expert_id => 1,
:photo => #provider.details.photo.url
}] }.to_json
end
it "should show return two providers" do
#provider2 = create(:provider)
get :index
response.body.gsub(/\s+/, "").should == {
:experts => [{
:availability => false,
:name => "#{#provider.user.first_name}#{#provider.user.last_name}",
:expert_id => 1,
:photo => #provider.details.photo.url
},
{
:availability => false,
:name => "#{#provider.user.first_name}#{#provider.user.last_name}",
:expert_id => 2,
:photo => #provider.details.photo.url
}
] }.to_json
end

Database cleaner is wrapping each of your specs in a transaction, and rolling back that transaction at the end of the spec to remove any database changes. Rolling back a transaction doesn't reset the autoincrement or sequence values used to auto assign the primary key though.
I'd strongly recommend not hardcoding the ids. You mention that your examples will get more complicated, in which case having random integers sprinkled through your code will be even less maintainable than in simpler examples. Assuming you're using mysql then using the truncation strategy will reset autoincrement values, but it is also a lot slower.

Is it the request/integration specs where the database is not cleaning properly? If so it's possibly because you're cleaning with a transaction strategy and not a truncation strategy.
Try adding this, which changes the strategy to truncation for integration specs only:
config.before type: :integration do
DatabaseCleaner.strategy = :truncation
end
You can't use a transaction strategy for integration specs because Capybara runs in a separate thread with a different database connection.
This article helped me a lot in setting up my DB stuff for rspec/capybara: Sane Rspec config for clean, and slightly faster, specs.

I ran into this a lot when starting testing. I eventually found that it was not due to the database cleaner issue (as I also suspected), but rather the structure of the code that does the tests.
The best way I can try and descibe is is to say that basically if you do things outside of your before(:each) set up blocks and the actual it and should it end up being 'outside' of the actual test and causes these issues.
Specifically I suspect that there could be a problem here:
it "should show return two providers" do
#provider2 = create(:provider)
get :index
response.body.gsub(/\s+/, "").should == {
I would look to change this to something more like:
it "should show return two providers" do
before(:each) do {
#provider2 = create(:provider)
get :index
}
response.body.gsub(/\s+/, "").should == {

Related

RSpec ActiveStorage: Error ActiveRecord::RecordNotFound: Couldn't find ActiveStorage::Blob with 'id'

Hi guys Can I please get some help to figure out what is going wrong with my RSpec test. I been looking all over the internet and haven't found anything that would point to reason why i'm getting this error.
What i'm trying to do is test that an instance variable has some data from the DB. The error is happening when the test reaches the Fabricator that creates the active record and also the ActiveStorage Blob. I have added the database-cleaner gem but not sure of something is messing up with that or i'm missing something when using RSpec, active storage, and DBcleaner.
What is strange is that I have another test that also creates the same Fabricated Object and I dont get the error you will see below. If I comment out the test below the other test runs just fine. Any help would be really appreciated. Been stuck on this for hours :$
UPDATE: I tried to look into what was happening after the graphic fabricator ran and when I looked into the attachments using the attached? methods each of the files were actually attached. All four files came back as attached true. I thought there would be something going on with the cleaner so I added to the rails_helper to purge all the files after each test is done.
config.after(:each) do
DatabaseCleaner.clean
ActiveStorage::Attachment.all.each { |attachment| attachment.purge }
end
This should grab all attachments and delete the attachment and blobs after each test runs. But this didnt do anything. Still not sure what is happening.
Any help would be appreciated.
Thanks :)
ERROR:
1) Dashboard::VendorDashboardsController GET sales Authenticated assigns instance variable with purchases for that store
Failure/Error: graphic = Fabricate(:graphic, store:store)
ActiveRecord::RecordNotFound:
Couldn't find ActiveStorage::Blob with 'id'=1
# ./spec/controllers/Dashboard/vendor_dashboards_controller_spec.rb:26:in `block (4 levels) in <top (required)>'
RSPEC TEST
it "assigns instance variable with purchases for that store" do
#user = Fabricate(:user)
vendor = Fabricate(:vendor, user:#user)
store = Fabricate(:store, vendor:vendor)
graphic = Fabricate(:graphic, store:store)
purchase_one = Fabricate(:purchase, purchasable_id:graphic.id)
purchase_two = Fabricate(:purchase, purchasable_id:graphic.id)
#user.add_role(:vendor)
login_user(#user)
get :sales
expect(assigns(:sales)).to eq(Purchase.where(store_id:store.id))
end
FABRICATOR
Fabricator(:graphic)do
store
name "Icon Set"
language "Spanish"
standard_license_price 10.00
business_license_price 100.00
reviewed true
status "Approved"
files_included "PSD, CSS"
category "Vectors"
subject "Characters"
support "3 Month Support"
layered true
high_resolution true
pixel_dimensions "2000x2000"
software_required "Illustrator"
thumbnail_image ActiveStorage::Blob.create_after_upload!(io: File.open(Rails.root.join('spec', 'support', 'assets', 'graphics', 'Advertise.png'), 'rb'), filename: 'Advertise.png',content_type: 'image/png').signed_id
main_image ActiveStorage::Blob.create_after_upload!(io: File.open(Rails.root.join('spec', 'support', 'assets', 'graphics', 'Chat.png'), 'rb'), filename: 'Chat.png',content_type: 'image/png').signed_id
product_images ActiveStorage::Blob.create_after_upload!(io: File.open(Rails.root.join('spec', 'support', 'assets', 'graphics', 'Chat.png'), 'rb'), filename: 'Chat.png',content_type: 'image/png').signed_id
graphic_asset_files ActiveStorage::Blob.create_after_upload!(io: File.open(Rails.root.join('spec', 'support', 'assets', 'graphics', 'test_illustrations.zip'), 'rb'), filename: 'test_illustrations.zip',content_type: 'application/zip').signed_id
description "This is the best graphic design"
end
RAILS HELPER FILE
# This file is copied to spec/ when you run 'rails generate rspec:install'
require 'spec_helper'
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../config/environment', __dir__)
# Prevent database truncation if the environment is production
abort("The Rails environment is running in production mode!") if Rails.env.production?
require 'rspec/rails'
begin
ActiveRecord::Migration.maintain_test_schema!
rescue ActiveRecord::PendingMigrationError => e
puts e.to_s.strip
exit 1
end
RSpec.configure do |config|
# Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
config.fixture_path = "#{::Rails.root}/spec/fixtures"
config.use_transactional_fixtures = false
config.infer_spec_type_from_file_location!
# Filter lines from Rails gems in backtraces.
config.filter_rails_from_backtrace!
config.include Rails.application.routes.url_helpers
config.include Sorcery::TestHelpers::Rails::Controller, type: :controller
config.include Sorcery::TestHelpers::Rails::Integration, type: :feature
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each, :js => true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
Shoulda::Matchers.configure do |config|
config.integrate do |with|
with.test_framework :rspec
with.library :rails
end
end
end
The problem with your factory is that property values are evaluated just once, at file load time. And then they're reused for all objects.
thumbnail_image ActiveStorage::Blob.create_after_upload!(...).signed_id
Use a block here
thumbnail_image { ActiveStorage::Blob.create_after_upload!(...).signed_id }
This way, for each new fabricated object you'll get its own blob.

How to proceed with flaky specs after successful bisect

Using Rails 5 and Rspec 3.7 I have a fairly simple test that is currently flapping (sometimes passing sometimes failing). Through the debugging I've done so far it seems that values i'm saving to my test database are not persisting between tests, but I can not figure out why this is the case.
Here is the test suite with comments on the flapping test (the rest consistently pass)
describe ResourceCenterController, type: :controller do
before(:each) do
#platform_instance = FactoryBot.create(:platform_instance)
#domain = FactoryBot.create(:domain, platform_instance: #platform_instance)
#user = FactoryBot.create(:user, platform_instance: #platform_instance, first_name: "O'flaggan")
end
context 'when user IS signed in' do
before(:each) do
login_user(#user)
end
context 'when user in ONE community' do
before(:each) do
#user.communities = [#platform_instance.communities.first]
#user.save!
end
describe '#index' do
before(:each) do
#rc = FactoryBot.create(:resource_center, platform_instance: #platform_instance, launch_at: nil, expire_at: nil)
end
context 'when community assigned NO resource centers' do
before(:each) do
#rc.communities = []
#rc.save!
get :index
end
it_behaves_like '200 w name in body' do
let(:names) { ['There are no files for your review at the moment.'] }
end
end
context 'when community assigned ONE resource center' do
before(:each) do
#rc.communities = [#user.communities.first]
#rc.save!
end
context 'when resource center assigned NO mediafiles' do
before(:each) do
#rc.mediafiles = []
#rc.save!
get :index
end
it_behaves_like '200 w name in body' do
let(:names) { ['There are no files for your review at the moment.'] }
end
end
# this test is flapping
# sometimes it will persist the mediafile and it will show up
# other times it will be saved, why is that?
context 'when resource center assigned ONE mediafile' do
before(:each) do
#mediafile = FactoryBot.create(:mediafile, platform_instance: #platform_instance)
#rc.mediafiles << #mediafile
#rc.save!
get :index
end
it_behaves_like '200 w name in body' do
let(:names) { ["#{#mediafile.name}"] }
end
end
end
end
end
end
end
Here is the shared context
shared_context '200 w name in body' do
it 'returns 200' do
expect(response.status).to eq(200)
end
it 'renders the view' do
names.each do |name|
expect(response.body).to include(name)
end
end
end
Edit: I learned of the bisect flag and ran it with this output
Bisect started using options: "spec/controllers/resource_center_controller_spec.rb"
Running suite to find failures... (7.39 seconds)
Starting bisect with 1 failing example and 5 non-failing examples.
Checking that failure(s) are order-dependent... failure appears to be order-dependent
Round 1: bisecting over non-failing examples 1-5 .. multiple culprits detected - splitting candidates (13.84 seconds)
Round 2: bisecting over non-failing examples 1-3 . ignoring examples 1-2 (6.95 seconds)
Round 3: bisecting over non-failing examples 4-5 . ignoring example 4 (6.75 seconds)
Bisect complete! Reduced necessary non-failing examples from 5 to 2 in 34.1 seconds.
The minimal reproduction command is:
rspec ./spec/controllers/resource_center_controller_spec.rb[1:1:1:1:2:1:1:1,1:1:1:1:2:2:1:1,1:1:1:1:2:2:1:2]
Edit: here is the factory for mediafile
FactoryBot.define do
# pi = PlatformInstance.select
factory :mediafile do
name { Faker::Simpsons.character }
platform_instance_uuid { PlatformInstance.first.uuid } # stick to platforminstance.first for now
platform_instance { PlatformInstance.first } # had tried to use a variable, but was
# not working
description { Faker::Simpsons.quote }
document { File.new("#{Rails.root}/spec/support/fixtures/mediafiles/document_01.pdf") }
image { File.new("#{Rails.root}/spec/support/fixtures/mediafiles/image_01.jpg") }
# review_with_mediafiles will create mediafile data after the review has been created
factory :mediafile_with_review do
after(:create) do |mediafile, evaluator|
create(:review, mediafile: mediafile)
end
end
end
end
and here is the factory for resource center
FactoryBot.define do
factory :resource_center do
title { Faker::Company.catch_phrase }
description { Faker::Lorem.paragraph(10) }
launch_at { Time.now }
expire_at { Time.now + 1.week }
platform_instance_uuid { PlatformInstance.first.uuid } # stick to PlatformInstance.first for now
platform_instance { PlatformInstance.first } # had tried to use a variable, but was
# not working
status { [:testing, :live].sample }
# review_with_mediafiles will create mediafile data after the review has been created
# this factory inherits everything from the factory it is nested under
factory :resource_center_with_mediafiles do
after(:create) do |resource_center, evaluator|
create(:mediafile, resource_centers: [resource_center])
end
end
end
end
The controller method itself is fairly simple
def index
#resource_centers = current_user.resource_centers.within_dates
end
current_user variable is assigned in the application controller which I don't think is super necessary to include here. The view is also fairly simple and can be seen below
-content_for :breadcrumbs do
=render 'layouts/shared/breadcrumbs', breadcrumbs: [link_to('Home', user_root_path), 'Resource Center']
-files_present = false
-#resource_centers.each do |resource_center|
-if resource_center.mediafiles.present?
-files_present = true
%h3.color-primary= resource_center.title.html_safe
=resource_center.description.html_safe
.space-above-2
-resource_center.mediafiles.sort.each do |mediafile|
=render 'resource_center/mediafile_item', resource_center: resource_center, mediafile: mediafile
-if !files_present
%h4 There are no files for your review at the moment.
Here is the partial rendered in the above view.
.index-list
.index-item.large-avatar
.item-avatar
=link_to resource_center_mediafile_view_path(resource_center, mediafile) do
= image_tag mediafile.image.url
.item-content
.item-header= mediafile.name
.item-attribute-list
%span.item-attribute
-if mediafile.duration.present?
%strong DURATION:
=pluralize(mediafile.duration, "minute")
-if mediafile.document.size.to_i > 0
%strong SIZE:
=number_to_human_size(mediafile.document.size)
.item-actions
-if resource_center.downloadable
=link_to 'Download', mediafile.download_url, class: 'mui-button default', target: '_blank'
=link_to 'View', resource_center_mediafile_view_path(resource_center, mediafile), class: 'mui-button'
Here is the spec_helper file:
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
RSpec.configure do |config|
config.before(:each) do
#only modify the request when testing controllers
if described_class <= ApplicationController
request.host = 'localhost:3000'
end
end
config.include Rails.application.routes.url_helpers
config.expect_with :rspec do |expectations|
expectations.include_chain_clauses_in_custom_matcher_descriptions = true
end
config.mock_with :rspec do |mocks|
mocks.verify_partial_doubles = true
end
config.before(:suite) do
DatabaseCleaner.strategy = :transaction
DatabaseCleaner.clean_with(:truncation)
end
config.around(:each) do |example|
DatabaseCleaner.cleaning do
example.run
end
end
config.before(:all) do
DatabaseCleaner.start
end
config.after(:all) do
DatabaseCleaner.clean
end
config.shared_context_metadata_behavior = :apply_to_host_groups
# config.include Rails.application.routes.url_helpers
end
Please let me know if there is other information that would be helpful. I think this is something wrong with my test suite, particularly the before(:each) blocks, but my experimentation has not given me any insights.
Disclaimer: I did not read the whole code you posted, so I won't give you the answer what causes this flakiness (or flappines as you call it) but I'll give you a method to find it yourself. (fish vs. fishing rod thing kinda thing)
Using bisect is great, and since it says that the issue is order dependent it's fairly easy to continue.
You can now set a breakpoint in the failing it and investigate why the results are different than expected. Most probably there's some leftover junk in the DB left from some other spec.
When you pinpoint the reason for the failing spec, you can run command:
rspec --format doc \
./spec/controllers/resource_center_controller_spec.rb[1:1:1:1:2:1:1:1,1:1:1:1:2:2:1:1,1:1:1:1:2:2:1:2]
This will tell you in what order the tests are run (since [1:1:1:1:2:1:1:1,1:1:1:1:2:2:1:1,1:1:1:1:2:2:1:2] is not very human friendly)
and you can look for the spec that leaves the "state unclean" (mentioned DB junk, but could be something else)
When you pin-point the offender you can add some crude fix (like Model.destroy_all after it, to confirm that it's The Reason).
Please note that this is not the proper fix yet.
After you confirm that this is true - you're ready to search for a solution. This can be using DBCleaner for your specs, or fixing some cache code that is misbehaving or something completely different (feel free to ask another question when you have the answers)
One extra note: in many projects order of the specs will be randomized. In such case bisecting will fail unless you know the --seed under which the specs fail.

Not using fixtures for specific unit tests under Rails 4.2

I have a few tests in my suite that absolutely need an empty database to run.
Thing is, rails is automatically loading the fixtures for every test, and I can't seem to find a way to have it not load for this specific test.
I could drop the database before each test, but that's slow and requires all other tests that require fixtures to reload the fixtures afterwards.
Is there any way to have a certain test class (i.e. "NoFixturesTest") not load the fixtures (or unload them), without breaking all other tests?
Thanks!
EDIT:
Thanks to the answer below, I made this:
module DisableFixturesHelper
def self.included(base)
base.setup :setup_drop_db
# Have a unit test double-check that the fixtures are really gone
base.test "0 fixtures not loaded" do
assert_equal 0, Table1.count, "Table1 isn't empty!"
#etc
end
end
# Call to reset the db and therefore disable fixtures during a single unit test
def setup_drop_db
Rails.logger.info { "Dropping database." }
DatabaseCleaner.clean_with :truncation
end
end
And then in the TestCases that require an empty database (no fixtures), I add
include DisableFixturesHelper
The problem with this is that this drops the database once per test, AFTER the fixtures have already been loaded, so it's really slow. It's literally reloading the fixtures before the start of each unit test, just to drop them.
Still, it works.
do you know DatabaseCleaner ???
maybe it can help, you can also define what don't delete
something like:
DatabaseCleaner.strategy = :truncation, {:except => %w[widgets]}
here is an initializer that I always use:
RSpec.configure do |config|
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each, :js => true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
DatabaseCleaner.start
end
config.append_after(:each) do
DatabaseCleaner.clean
end
end

Rspec Integration Test not Cleaning the Database

The database is not being cleaned after each integration test. The value stays in the database.
Is there an option I should have to make this happen?
Thanks
I think https://github.com/bmabey/database_cleaner is what you need.
For anyone using before(:all) hooks, be aware that these hooks are executed before the transaction associated to the fixture is opened. This means that any data created by before(:all) hooks will not be rolled back by transactional fixtures. You can read more in the RSpec documentation.
I just wanted to mention this because I was bit by it and my initial instinct was to jump to Database Cleaner (which wound up not being needed and eventually not working).
How do I prepare test database(s) for Rails rspec tests without running rake spec?
My answer there might be of interest to you. it's a nice solution. For your case, you would probably need something like
config.after :each do
ActiveRecord::Base.subclasses.each(&:delete_all)
end
Look here for a tutorial: http://railscasts.com/episodes/257-request-specs-and-capybara
It describes Database Cleaner besides Rspec and Capybara
You want DatabaseCleaner, but you may find that the :truncation strategy is a bit too slow to run all the time. It's really only necessary for integration tests, so you can do this:
# spec/spec_helper.rb
require 'database_cleaner'
config.before(:suite) do
DatabaseCleaner.clean_with :truncation
DatabaseCleaner.strategy = :transaction
end
config.before(:each) do |group|
# The strategy needs to be set before we call DatabaseCleaner.start
case group.example.metadata[:type]
when :feature
DatabaseCleaner.strategy = :truncation
else
DatabaseCleaner.strategy = :transaction
end
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
# spec/features/your_feature_spec.rb
require 'spec_helper'
describe "An integration test", :type => :feature do
end
# spec/model/your_model_spec.rb
require 'spec_helper'
describe "A unit test" do
end
Obviously, this only applies if you're doing integration tests with RSpec directly vs. doing them with Cucumber.
There are two ways to accomplish this:
Configure transactional examples for each individual test.
Configure transactional examples for all the tests.
If you opt for option 1: At the top of the spec file, after:
require 'spec_helper'
Add:
RSpec.configure {|c| c.use_transactional_examples = true }
That will roll back the transactions after each example.
2.If you want to configure it globally, then, in the spec_helper.rb
RSpec.configure do |config|
...
config.use_transactional_examples = true # Add this
...
end

rails test database won't wipe

I'm running rails 3.0.3 and using rspec-rails 2.4.1 with a postgresql database. Whenever I run my RSpec tests, the data remains at the end. Does anyone know how to get rails or rspec to wipe the test environment's data between each use?
Please tell me if there's any further information that could make answering my question easier.
Thanks!Tristan
Install the database_cleaner gem and then add this to your spec_helper.rb.
Spec::Runner.configure do |config|
config.before(:suite) do
DatabaseCleaner.strategy = :transaction
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
end
Use transactional examples to rollback the data after every test run
RSpec.configure do |config|
config.use_transactional_examples = true
end
You don't need any extra gem in order to clean your test DB between runs. In your spec_helper.rb file, configure rspec as follows:
RSpec.configure do |c|
c.around(:each) do |example|
ActiveRecord::Base.connection.transaction do
example.run
raise ActiveRecord::Rollback
end
end
end
Another possibility, that I just put myself through, is using the wrong before block.
I accidentally set a before block as an all instead of an each:
before :all do
user = FactoryGirl.create(:user)
sign_in user
end
This caused the user to stick around in the database for the entire rspec run, which caused validation collisions.
Instead, the before should be an each so that everything is kept clean through the rspec run:
before :each do
user = FactoryGirl.create(:user)
sign_in user
end
If you've made this mistake, then you will probably need to manually clean up your test database before things go back to normal. The simplest way to do that is probably to truncate each of the tables (aside from schema_migrations).

Resources