I'm trying to write some tests involving file operations. I want to use some fake file system (something like VCR for external services) and I have found fakeFS. Unfortunately, either I can't set it right or something is broken (which I doubt, it's quite basic function), I've prepared simple example which illustrates what I mean, let the code speak:
With real FS:
module MyModule
describe Something do
before(:all) do
File.open("#{Rails.root}/foo.txt", 'w+') { |f| f.write 'content'}
end
it 'should exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
it 'should still exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
end
end
Running that gives:
bash-4.2$ rspec
..
Finished in 0.00161 seconds
2 examples, 0 failures
Adding fakeFS in such way:
require 'fakefs/spec_helpers'
module MyModule
describe Something do
include FakeFS::SpecHelpers
FakeFS.activate!
FakeFS::FileSystem.clone(Rails.root)
before(:all) do
File.open("#{Rails.root}/foo.txt", 'w+') { |f| f.write 'content'}
end
it 'should exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
it 'should still exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
end
end
results in:
bash-4.2$ rspec
.F
Failures:
1) MyModule::Something should still exist
Failure/Error: expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
expected: true value
got: false
# ./spec/models/something_spec.rb:23:in `block (2 levels) in <module:MyModule>'
Finished in 0.00354 seconds
2 examples, 1 failure
So it seems like file is not persisted through subsequent tests. Do I misunderstand how before(:all) works or do I do something wrong? If so then why that code works with real files?
If it is 'not a bug, just a feature' then is there any other fake filesystem gem which is consistent with real one? Or do I have to stay with real files to get tests that.. well, test?
I found the answer just after creating that question, duh ;) I've looked into source of that lib and found suspicious line.
Instead of FakeFS::SpecHelpers I've included FakeFS::SpecHelpers::All which is the same code except FakeFS::FileSystem is not being cleared after each call, now it behaves correctly.
Related
I have a simple test but the describe keyword is not working in Sorbet tests.
The error I'm receiving on these methods:
Method `describe` does not exist on `T.class_of(<root>)`7003
RSpec.describe(Model) do
describe 'my test' do
before(:each) do # .before error
user = FactoryBot.create(:user)
end
it 'can fill in all fields' do # .it errors
end
end
end
I think I need to tell Sorbet some how that this is called in the context of spec_helper.rbbut I'm not sure how to do that.
I've already installed this gem rspec-sorbet and ran
spec/spec_helper.rb
require 'rspec/sorbet'
To silence the errors, I ran this:
RSpec.describe(Model) do
T.bind(self, T.untyped)
# T.bind(self, RSpec) This does not work either
end
Using Rails 5 and Rspec 3.7 I have a fairly simple test that is currently flapping (sometimes passing sometimes failing). Through the debugging I've done so far it seems that values i'm saving to my test database are not persisting between tests, but I can not figure out why this is the case.
Here is the test suite with comments on the flapping test (the rest consistently pass)
describe ResourceCenterController, type: :controller do
before(:each) do
#platform_instance = FactoryBot.create(:platform_instance)
#domain = FactoryBot.create(:domain, platform_instance: #platform_instance)
#user = FactoryBot.create(:user, platform_instance: #platform_instance, first_name: "O'flaggan")
end
context 'when user IS signed in' do
before(:each) do
login_user(#user)
end
context 'when user in ONE community' do
before(:each) do
#user.communities = [#platform_instance.communities.first]
#user.save!
end
describe '#index' do
before(:each) do
#rc = FactoryBot.create(:resource_center, platform_instance: #platform_instance, launch_at: nil, expire_at: nil)
end
context 'when community assigned NO resource centers' do
before(:each) do
#rc.communities = []
#rc.save!
get :index
end
it_behaves_like '200 w name in body' do
let(:names) { ['There are no files for your review at the moment.'] }
end
end
context 'when community assigned ONE resource center' do
before(:each) do
#rc.communities = [#user.communities.first]
#rc.save!
end
context 'when resource center assigned NO mediafiles' do
before(:each) do
#rc.mediafiles = []
#rc.save!
get :index
end
it_behaves_like '200 w name in body' do
let(:names) { ['There are no files for your review at the moment.'] }
end
end
# this test is flapping
# sometimes it will persist the mediafile and it will show up
# other times it will be saved, why is that?
context 'when resource center assigned ONE mediafile' do
before(:each) do
#mediafile = FactoryBot.create(:mediafile, platform_instance: #platform_instance)
#rc.mediafiles << #mediafile
#rc.save!
get :index
end
it_behaves_like '200 w name in body' do
let(:names) { ["#{#mediafile.name}"] }
end
end
end
end
end
end
end
Here is the shared context
shared_context '200 w name in body' do
it 'returns 200' do
expect(response.status).to eq(200)
end
it 'renders the view' do
names.each do |name|
expect(response.body).to include(name)
end
end
end
Edit: I learned of the bisect flag and ran it with this output
Bisect started using options: "spec/controllers/resource_center_controller_spec.rb"
Running suite to find failures... (7.39 seconds)
Starting bisect with 1 failing example and 5 non-failing examples.
Checking that failure(s) are order-dependent... failure appears to be order-dependent
Round 1: bisecting over non-failing examples 1-5 .. multiple culprits detected - splitting candidates (13.84 seconds)
Round 2: bisecting over non-failing examples 1-3 . ignoring examples 1-2 (6.95 seconds)
Round 3: bisecting over non-failing examples 4-5 . ignoring example 4 (6.75 seconds)
Bisect complete! Reduced necessary non-failing examples from 5 to 2 in 34.1 seconds.
The minimal reproduction command is:
rspec ./spec/controllers/resource_center_controller_spec.rb[1:1:1:1:2:1:1:1,1:1:1:1:2:2:1:1,1:1:1:1:2:2:1:2]
Edit: here is the factory for mediafile
FactoryBot.define do
# pi = PlatformInstance.select
factory :mediafile do
name { Faker::Simpsons.character }
platform_instance_uuid { PlatformInstance.first.uuid } # stick to platforminstance.first for now
platform_instance { PlatformInstance.first } # had tried to use a variable, but was
# not working
description { Faker::Simpsons.quote }
document { File.new("#{Rails.root}/spec/support/fixtures/mediafiles/document_01.pdf") }
image { File.new("#{Rails.root}/spec/support/fixtures/mediafiles/image_01.jpg") }
# review_with_mediafiles will create mediafile data after the review has been created
factory :mediafile_with_review do
after(:create) do |mediafile, evaluator|
create(:review, mediafile: mediafile)
end
end
end
end
and here is the factory for resource center
FactoryBot.define do
factory :resource_center do
title { Faker::Company.catch_phrase }
description { Faker::Lorem.paragraph(10) }
launch_at { Time.now }
expire_at { Time.now + 1.week }
platform_instance_uuid { PlatformInstance.first.uuid } # stick to PlatformInstance.first for now
platform_instance { PlatformInstance.first } # had tried to use a variable, but was
# not working
status { [:testing, :live].sample }
# review_with_mediafiles will create mediafile data after the review has been created
# this factory inherits everything from the factory it is nested under
factory :resource_center_with_mediafiles do
after(:create) do |resource_center, evaluator|
create(:mediafile, resource_centers: [resource_center])
end
end
end
end
The controller method itself is fairly simple
def index
#resource_centers = current_user.resource_centers.within_dates
end
current_user variable is assigned in the application controller which I don't think is super necessary to include here. The view is also fairly simple and can be seen below
-content_for :breadcrumbs do
=render 'layouts/shared/breadcrumbs', breadcrumbs: [link_to('Home', user_root_path), 'Resource Center']
-files_present = false
-#resource_centers.each do |resource_center|
-if resource_center.mediafiles.present?
-files_present = true
%h3.color-primary= resource_center.title.html_safe
=resource_center.description.html_safe
.space-above-2
-resource_center.mediafiles.sort.each do |mediafile|
=render 'resource_center/mediafile_item', resource_center: resource_center, mediafile: mediafile
-if !files_present
%h4 There are no files for your review at the moment.
Here is the partial rendered in the above view.
.index-list
.index-item.large-avatar
.item-avatar
=link_to resource_center_mediafile_view_path(resource_center, mediafile) do
= image_tag mediafile.image.url
.item-content
.item-header= mediafile.name
.item-attribute-list
%span.item-attribute
-if mediafile.duration.present?
%strong DURATION:
=pluralize(mediafile.duration, "minute")
-if mediafile.document.size.to_i > 0
%strong SIZE:
=number_to_human_size(mediafile.document.size)
.item-actions
-if resource_center.downloadable
=link_to 'Download', mediafile.download_url, class: 'mui-button default', target: '_blank'
=link_to 'View', resource_center_mediafile_view_path(resource_center, mediafile), class: 'mui-button'
Here is the spec_helper file:
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
RSpec.configure do |config|
config.before(:each) do
#only modify the request when testing controllers
if described_class <= ApplicationController
request.host = 'localhost:3000'
end
end
config.include Rails.application.routes.url_helpers
config.expect_with :rspec do |expectations|
expectations.include_chain_clauses_in_custom_matcher_descriptions = true
end
config.mock_with :rspec do |mocks|
mocks.verify_partial_doubles = true
end
config.before(:suite) do
DatabaseCleaner.strategy = :transaction
DatabaseCleaner.clean_with(:truncation)
end
config.around(:each) do |example|
DatabaseCleaner.cleaning do
example.run
end
end
config.before(:all) do
DatabaseCleaner.start
end
config.after(:all) do
DatabaseCleaner.clean
end
config.shared_context_metadata_behavior = :apply_to_host_groups
# config.include Rails.application.routes.url_helpers
end
Please let me know if there is other information that would be helpful. I think this is something wrong with my test suite, particularly the before(:each) blocks, but my experimentation has not given me any insights.
Disclaimer: I did not read the whole code you posted, so I won't give you the answer what causes this flakiness (or flappines as you call it) but I'll give you a method to find it yourself. (fish vs. fishing rod thing kinda thing)
Using bisect is great, and since it says that the issue is order dependent it's fairly easy to continue.
You can now set a breakpoint in the failing it and investigate why the results are different than expected. Most probably there's some leftover junk in the DB left from some other spec.
When you pinpoint the reason for the failing spec, you can run command:
rspec --format doc \
./spec/controllers/resource_center_controller_spec.rb[1:1:1:1:2:1:1:1,1:1:1:1:2:2:1:1,1:1:1:1:2:2:1:2]
This will tell you in what order the tests are run (since [1:1:1:1:2:1:1:1,1:1:1:1:2:2:1:1,1:1:1:1:2:2:1:2] is not very human friendly)
and you can look for the spec that leaves the "state unclean" (mentioned DB junk, but could be something else)
When you pin-point the offender you can add some crude fix (like Model.destroy_all after it, to confirm that it's The Reason).
Please note that this is not the proper fix yet.
After you confirm that this is true - you're ready to search for a solution. This can be using DBCleaner for your specs, or fixing some cache code that is misbehaving or something completely different (feel free to ask another question when you have the answers)
One extra note: in many projects order of the specs will be randomized. In such case bisecting will fail unless you know the --seed under which the specs fail.
From time to time, some specs of my Rails app seem to fail randomly because suddenly English is not the default language anymore, but German:
expected: "Project test customer - Project test name (Audit, Access for all, 2015-06-15).pdf"
got: "Project test customer - Project test name (Audit, Zugang für alle, 2015-06-15).pdf"
As one can see, the part "Access for all" suddenly is "Zugang für alle". I googled for a solution and it seems that I18n.locale is a global object, so when it's changed in a spec, it persists.
The problem doesn't always occur, but I can reproduce it when specifying a seed like this: rspec --seed 51012. So it really seems to be some issue with a spec being executed before (or after) some other spec.
I have a feature spec which tests whether the locale can be changed, like this:
it 'offers contents in german' do
visit root_path(locale: :de)
expect(page).to have_content 'Willkommen'
end
I suspect this could be the problematic spec, and when it's run early, it has impact on other specs.
I hoped that I could solve it by setting the language in the spec back to the default like this:
it 'offers contents in german' do
visit root_path(locale: :de)
expect(page).to have_content 'Willkommen'
I18n.locale = :en
end
Didn't work, also this one didn't:
it 'offers contents in german' do
visit root_path(locale: :de)
expect(page).to have_content 'Willkommen'
visit root_path(locale: :en)
end
I'm a bit clueless now. How could I debug the situation, so I definitely find the source of the problem and fix it (or at least work around it)?
Update
Using Dave's answer (rspec --bisect) I found the problem.
# application_controller_spec.rb
describe ApplicationController do
controller(ApplicationController) do
def index
render text: 'Hello World'
end
end
describe 'locale parameter' do
it 'is set to english when not available in the request' do
get :index
expect(I18n.locale).to eq :en
end
it 'can be set through the request' do
get :index, locale: :de
expect(I18n.locale).to eq :de
end
end
end
Depending on the order that these specs were run, the locale was set to :de or :en for following specs.
I fixed it with BoraMa's suggested code snippet:
RSpec.configure do |config|
config.after(:each) { I18n.locale = :en }
end
I'm still a bit astonished that RSpec/Rails don't do this on their own automatically...
Comment out the spec that you suspect and run rspec --seed 51012. If that passes, you know the culprit.
If not, this looks like a job for rspec --bisect. Do
rspec --seed 51012 --bisect
and let RSpec find the minimal set of examples (probably 2) that reproduce the failure. You can then decide what to do about the culprit.
This feature is available in RSpec 3.3 and better in RSpec 3.4. More here: https://relishapp.com/rspec/rspec-core/docs/command-line/bisect
I am working on a Rails application. I am trying to stub a method globally.
What I am doing is to stub it inside the RSpec configuration, on a before(:suite) block as follows:
RSpec.configure do |config|
config.before(:suite) do
allow_any_instance_of(MyModel).to receive(:my_method).and_return(false)
end
end
However, starting the test fails with the following error:
in `method_missing': undefined method `allow_any_instance_of' for #<RSpec::Core::ExampleGroup:0x00000008d6be08> (NoMethodError)
Any clue? How should I stub a method globally using RSpec?
P.
It probably is a context / initialization issue. Doing it in config.before(:each) should solve your problem.
Do not stub methods in before(:suite) because stubs are cleared after each example, as stated in the rspec-mocks README:
Use before(:each), not before(:all)
Stubs in before(:all) are not supported. The reason is that all stubs
and mocks get cleared out after each example, so any stub that is set
in before(:all) would work in the first example that happens to run in
that group, but not for any others.
Instead of before(:all), use before(:each).
I think that's why allow_any_instance_of is not available in before(:suite) block, but is available in before(:each) block.
If the method is still missing, maybe you configured rspec-mocks to only allow :should syntax. allow_any_instance_of was introduced in RSpec 2.14 with all the new :expect syntax for message expectations.
Ensure that this syntax is enabled by inspecting the value of RSpec::Mocks.configuration.syntax. It is an array of the available syntaxes in rspec-mocks. The available syntaxes are :expect and :should.
RSpec.configure do |config|
config.mock_with :rspec do |mocks|
mocks.syntax = [:expect, :should]
end
end
Once configured properly, you should be able to use allow_any_instance_of.
I recently ran into a case where I needed to stub something in a before(:all) or before(:context) block, and found the solutions here to not work for my use case.
RSpec docs on before() & after() hooks says that it's not supported:
before and after hooks can be defined directly in the example groups they
should run in, or in a global RSpec.configure block.
WARNING: Setting instance variables are not supported in before(:suite).
WARNING: Mocks are only supported in before(:example).
Note: the :example and :context scopes are also available as :each and
:all, respectively. Use whichever you prefer.
Problem
I was making a gem for writing a binary file format which contained at unix epoch timestamp within it's binary header. I wanted to write RSpec tests to check the output file header for correctness, and compare it to a test fixture binary reference file. In order to create fast tests I needed to write the file out once before all the example group blocks would run. In order to check the timestamp against the reference file, I needed to force Time.now() to return a constant value. This led me down the path of trying to stub Time.now to return my target value.
However, since rspec/mocks did not support stubbing within a before(:all) or before(:context) block it didn't work. Writing the file before(:each) caused other strange problems.
Luckily, I stumbled across issue #240 of rspec-mocks which had the solution!
Solution
Since January 9th 2014 (rspec-mocks PR #519) RSpec now contains a method to work around this:
RSpec::Mocks.with_temporary_scope
Example
require 'spec_helper'
require 'rspec/mocks'
describe 'LZOP::File' do
before(:all) {
#expected_lzop_magic = [ 0x89, 0x4c, 0x5a, 0x4f, 0x00, 0x0d, 0x0a, 0x1a, 0x0a ]
#uncompressed_file_data = "Hello World\n" * 100
#filename = 'lzoptest.lzo'
#test_fixture_path = File.join(File.dirname(__FILE__), '..', 'fixtures', #filename + '.3')
#lzop_test_fixture_file_data = File.open( #test_fixture_path, 'rb').read
#tmp_filename = File.basename(#filename)
#tmp_file_path = File.join( '', 'tmp', #tmp_filename)
# Stub calls to Time.now() with our fake mtime value so the mtime_low test against our test fixture works
# This is the mtime for when the original uncompressed test fixture file was created
#time_now = Time.at(0x544abd86)
}
context 'when given a filename, no options and writing uncompressed test data' do
describe 'the output binary file' do
before(:all) {
RSpec::Mocks.with_temporary_scope do
allow(Time).to receive(:now).and_return(#time_now)
# puts "TIME IS: #{Time.now}"
# puts "TIME IS: #{Time.now.to_i}"
my_test_file = LZOP::File.new( #tmp_file_path )
my_test_file.write( #uncompressed_file_data )
#test_file_data = File.open( #tmp_file_path, 'rb').read
end
}
it 'has the correct magic bits' do
expect( #test_file_data[0..8].unpack('C*') ).to eq #expected_lzop_magic
end
## [...SNIP...] (Other example blocks here)
it 'has the original file mtime in LZO file header' do
# puts "time_now= #{#time_now}"
if #test_file_data[17..21].unpack('L>').first & LZOP::F_H_FILTER == 0
mtime_low_start_byte=25
mtime_low_end_byte=28
mtime_high_start_byte=29
mtime_high_end_byte=32
else
mtime_low_start_byte=29
mtime_low_end_byte=32
mtime_high_start_byte=33
mtime_high_end_byte=36
end
# puts "start_byte: #{start_byte}"
# puts "end_byte: #{end_byte}"
# puts "mtime_low: #{#test_file_data[start_byte..end_byte].unpack('L>').first.to_s(16)}"
# puts "test mtime: #{#lzop_test_fixture_file_data[start_byte..end_byte].unpack('L>').first.to_s(16)}"
mtime_low = #test_file_data[mtime_low_start_byte..mtime_low_end_byte].unpack('L>').first
mtime_high = #test_file_data[mtime_high_start_byte..mtime_high_end_byte].unpack('L>').first
# The testing timestamp has no high bits, so this test should pass:
expect(mtime_low).to eq #time_now.to_i
expect(mtime_high).to eq 0
expect(mtime_low).to eq #lzop_test_fixture_file_data[mtime_low_start_byte..mtime_low_end_byte].unpack('L>').first
expect(mtime_high).to eq #lzop_test_fixture_file_data[mtime_high_start_byte..mtime_high_end_byte].unpack('L>').first
mtime_fixed = ( mtime_high << 16 << 16 ) | mtime_low
# puts "mtime_fixed: #{mtime_fixed}"
# puts "mtime_fixed: #{mtime_fixed.to_s(16)}"
expect(mtime_fixed).to eq #time_now.to_i
end
end
end
end
If you want a particular method to behave a certain way for your entire test suite, there's no reason to even deal with RSpec's stubs. Instead, you can simply (re)define the method to behave how you want in your test environment:
class MyModel
def my_method
false
end
end
This could go in spec/spec_helper.rb or a similar file.
What version of RSpec are you using? I believe allow_any_instance_of was introduced in RSpec 2.14. For earlier versions, you can use:
MyModel.any_instance.stub(:my_method).and_return(false)
You may use the following to stub a method 'do_this' of class 'Xyz' :
allow_any_instance_of(Xyz).to receive(:do_this).and_return(:this_is_your_stubbed_output)
This stubs the output to - ':this_is_your_stubbed_output' from wherever this function is invoked.
You may use the above piece of code in before(:each) block to make this applicable for all your spec examples.
Well, I'm doing some testing right with Rails+Rspec+Shoulda.
When I do a test like the following:
context #user do
describe 'Validation' do
describe :name
it { should allow_value('something').for :name }
end
end
end
When it fails, Rspec just output:
1) Validation name Valid
Failure/Error: it { should allow_value(value).for :name }
Did not expect errors when name is set to "something", got error:
# ./spec/models/user_spec.rb:4:in `block (3 levels) in <top (required)>'
It even says got error: but it doesn't output it! I actually know there is a validation error there, but I want Rspec to tell me that, how I would know what is failing to validate then?
What am I doing wrong? Is that the expected behaviour? I have to overwrite the helpers?
I dug into the Shoulda code and I found that it doesn't show the errors when checking for positive assert. But them are loaded into the #errors variable. So I just monkey patched the one method that defines the output:
module Shoulda
module ActiveRecord
module Matchers
def failure_message
"Did not expect #{expectation}, got error: \n#{#expected_message ? #matched_error : #errors.join("\n ")}"
end
end
end
end
The original said:
"Did not expect #{expectation}, got error: #{#matched_error}"
I saved it to /lib/shoulda/activerecord/matchers.rb and loaded it with config.autoload_paths += Dir["#{config.root}/lib/**/"]
Hope this helps someone with the same issue ^^
Yup welcome to spec testing so you need to recreate the error in console if you want the error, rspec is not a debugger just a test suite.
I run into this a lot