From time to time, some specs of my Rails app seem to fail randomly because suddenly English is not the default language anymore, but German:
expected: "Project test customer - Project test name (Audit, Access for all, 2015-06-15).pdf"
got: "Project test customer - Project test name (Audit, Zugang für alle, 2015-06-15).pdf"
As one can see, the part "Access for all" suddenly is "Zugang für alle". I googled for a solution and it seems that I18n.locale is a global object, so when it's changed in a spec, it persists.
The problem doesn't always occur, but I can reproduce it when specifying a seed like this: rspec --seed 51012. So it really seems to be some issue with a spec being executed before (or after) some other spec.
I have a feature spec which tests whether the locale can be changed, like this:
it 'offers contents in german' do
visit root_path(locale: :de)
expect(page).to have_content 'Willkommen'
end
I suspect this could be the problematic spec, and when it's run early, it has impact on other specs.
I hoped that I could solve it by setting the language in the spec back to the default like this:
it 'offers contents in german' do
visit root_path(locale: :de)
expect(page).to have_content 'Willkommen'
I18n.locale = :en
end
Didn't work, also this one didn't:
it 'offers contents in german' do
visit root_path(locale: :de)
expect(page).to have_content 'Willkommen'
visit root_path(locale: :en)
end
I'm a bit clueless now. How could I debug the situation, so I definitely find the source of the problem and fix it (or at least work around it)?
Update
Using Dave's answer (rspec --bisect) I found the problem.
# application_controller_spec.rb
describe ApplicationController do
controller(ApplicationController) do
def index
render text: 'Hello World'
end
end
describe 'locale parameter' do
it 'is set to english when not available in the request' do
get :index
expect(I18n.locale).to eq :en
end
it 'can be set through the request' do
get :index, locale: :de
expect(I18n.locale).to eq :de
end
end
end
Depending on the order that these specs were run, the locale was set to :de or :en for following specs.
I fixed it with BoraMa's suggested code snippet:
RSpec.configure do |config|
config.after(:each) { I18n.locale = :en }
end
I'm still a bit astonished that RSpec/Rails don't do this on their own automatically...
Comment out the spec that you suspect and run rspec --seed 51012. If that passes, you know the culprit.
If not, this looks like a job for rspec --bisect. Do
rspec --seed 51012 --bisect
and let RSpec find the minimal set of examples (probably 2) that reproduce the failure. You can then decide what to do about the culprit.
This feature is available in RSpec 3.3 and better in RSpec 3.4. More here: https://relishapp.com/rspec/rspec-core/docs/command-line/bisect
Related
I've created a 'lil helper file to DRY up my specs:
module RemoteMacros
def go_to_another_localhost(port)
Capybara.current_driver = :selenium
Capybara.app_host = "localhost:#{port}"
Capybara.run_server = false
end
def go_webapp
Capybara.current_driver = :rack_test
Capybara.app_host = "localhost:3000"
Capybara.run_server = true
end
end
And here's the spec that goes with it:
require "rails_helper"
describe "Placing an ad", :type => :feature do
it "(1) The ad token should match across domains" do
go_to_another_localhost(8000)
visit "/"
expect(page).to have_css( 'iframe[data-nc-token="demo-token"]' )
end
it "(2) can go places" do
go_webapp
visit "/"
save_and_open_page
expect(page).to have_content( 'lol' )
end
end
At first glance, it seems as though everything should work as expected (how many times has THAT been said? ;D ). However, my second test returns red, with a rather peculiar message:
Placing an ad
(1) The ad token should match across domains
(2) can go places (FAILED - 1)
Failures:
1) Placing an ad (2) can go places
Failure/Error: visit "/"
NoMethodError:
undefined method `empty?' for nil:NilClass
What I'm working on is some cross-domain JS. I need to verify that my JS in another domain matches an attribute of a model in my actual Rails app. I got the first part to work, where I leave my app to go to the other domain, but the second part blows up in my face, when I'm returning the app_host to the original location.
How I should I modify my code to accomplish my intention?
I have the following Cucumber feature testing an input form using typeahead.js:
#javascript
Scenario: Creating a battery using typeahead
When I create a new battery using typeahead
Then I should be on the show battery page
And I should see the battery created message
The test fails on the second step with the following error message:
ActiveRecord::RecordNotFound (ActiveRecord::RecordNotFound)
./features/step_definitions/admin/car_part_steps/battery_steps.rb:37:in `/^I should be on the show battery page$/'
features/admin/creating_car_parts/creating_batteries.feature:20:in `Then I should be on the show battery page'
The relevant step definitions are as follows:
When /^I create a new battery using typeahead$/ do
select_from_typeahead :field => 'battery_manufacturer_typeahead',
:select => #manufacturer.name
fill_in 'Type', :with => '700W'
click_button 'Create Battery'
end
Then /^I should be on the show battery page$/ do
battery = Battery.find_by_type_and_manufacturer_id!('700W', #manufacturer.id)
current_path.should == admin_battery_path(battery)
page.should have_content(battery.type)
end
The select_from_typeahead function is as follows:
def select_from_typeahead(params)
params[:js_field] ||= params[:field]
params[:select_typeahead] ||= params[:select]
fill_in params[:field], :with => params[:select][0, 2]
page.execute_script "$('##{params[:js_field]}').trigger('focus')"
page.execute_script "$('##{params[:js_field]}').trigger('keydown')"
sleep 0.5
page.execute_script "$('.tt-suggestion:contains(\"#{params[:select_typeahead]}\")').trigger('mouseenter').trigger('click')"
end
The problem appears not to have anything to do with the typeahead itself however, as the code works in the browser, and if I add some debug output, I notice that the battery gets saved to the database in the first step when running the test as well, it just mysteriously disappears before the second step runs.
I think it's an issue with database_cleaner, as I know that doesn't play nice with Javascript when set to use transactions, but I've already tried setting it to use truncation instead and disabled transactional fixtures and it still doesn't work.
My features/support/env.rb currently looks like this:
require 'simplecov'
SimpleCov.start 'rails'
require 'cucumber/rails'
Capybara.default_selector = :css
Capybara.javascript_driver = :webkit
ActionController::Base.allow_rescue = false
Cucumber::Rails::World.use_transactional_fixtures = false
DatabaseCleaner.strategy = :truncation
Cucumber::Rails::Database.javascript_strategy = :truncation
My environment is as follows:
rails 4.0.2
cucumber 1.3.10
cucumber-rails 1.4.0
capybara 2.2.0
capybara-webkit 1.1.0
database_cleaner 1.2.0
Am I missing something, is there some other way database_cleaner might still interfere with my test, or is it something else entirely that I haven't thought of?
Any ideas would be very welcome!
I don't think it has to do with Database Cleaner. That wouldn't do cleanup until the end of your scenario. To debug, I highly recommend installing Capybara Screenshot so that you can see exactly what is going on in your page. It provides you the method screenshot_and_open_image, which you can use to pop open an image of what the browser looks like at that exact moment. My guesses are that:
Your jQuery isn't doing what you expect because of timing issues - have you tried longer pauses?
Your transaction isn't committed. Have you looked in test.log to see if the transaction was committed?
Is your form failing validation? Try putting a screenshot_and_open_image at the beginning of your I should be on the show battery page step to see. Another thing I use is a shared step like this, so that I can add screenshots in my scenarios for debugging:
And /^I take a screenshot$/ do
Capybara::Screenshot.screenshot_and_open_image
end
I'm trying to write some tests involving file operations. I want to use some fake file system (something like VCR for external services) and I have found fakeFS. Unfortunately, either I can't set it right or something is broken (which I doubt, it's quite basic function), I've prepared simple example which illustrates what I mean, let the code speak:
With real FS:
module MyModule
describe Something do
before(:all) do
File.open("#{Rails.root}/foo.txt", 'w+') { |f| f.write 'content'}
end
it 'should exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
it 'should still exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
end
end
Running that gives:
bash-4.2$ rspec
..
Finished in 0.00161 seconds
2 examples, 0 failures
Adding fakeFS in such way:
require 'fakefs/spec_helpers'
module MyModule
describe Something do
include FakeFS::SpecHelpers
FakeFS.activate!
FakeFS::FileSystem.clone(Rails.root)
before(:all) do
File.open("#{Rails.root}/foo.txt", 'w+') { |f| f.write 'content'}
end
it 'should exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
it 'should still exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
end
end
results in:
bash-4.2$ rspec
.F
Failures:
1) MyModule::Something should still exist
Failure/Error: expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
expected: true value
got: false
# ./spec/models/something_spec.rb:23:in `block (2 levels) in <module:MyModule>'
Finished in 0.00354 seconds
2 examples, 1 failure
So it seems like file is not persisted through subsequent tests. Do I misunderstand how before(:all) works or do I do something wrong? If so then why that code works with real files?
If it is 'not a bug, just a feature' then is there any other fake filesystem gem which is consistent with real one? Or do I have to stay with real files to get tests that.. well, test?
I found the answer just after creating that question, duh ;) I've looked into source of that lib and found suspicious line.
Instead of FakeFS::SpecHelpers I've included FakeFS::SpecHelpers::All which is the same code except FakeFS::FileSystem is not being cleared after each call, now it behaves correctly.
I'm writing a typical test in my application where I create a model through a form and check that the model count equals 1.
The test fails because there are already multiple records in the test DB, and this count increases each time I run my tests. It looks like each example isn't happening inside a transaction (being rolled back) like it's supposed to, and I don't know why.
I have this line in my spec_helper.rb file, which is supposed to run each example in a transaction:
config.use_transactional_fixtures = true
Here is my spec that keeps generating model objects:
require 'spec_helper'
describe "Admin artwork pages" do
subject { page }
let(:gallery) { FactoryGirl.create(:gallery) }
describe "artwork creation" do
context "with valid attributes" do
it "creates new artwork" do
visit admin_gallery_artworks_path(gallery_id: gallery.id)
click_link 'Add new artwork'
fill_in 'artwork_title', with: 'Still Life'
click_button 'Create Artwork'
page.should have_text 'Successfully created'
Artwork.count.should eq 1
end
end
end
end
Here's the error message from Rspec:
Failures:
1) Admin artwork pages artwork creation with valid attributes creates new artwork
Failure/Error: Artwork.count.should eq 1
expected: 1
got: 153
(compared using ==)
Edit: Contents of my spec_helper.rb file:
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
require 'rspec/autorun'
require 'capybara/rails'
require 'capybara/rspec'
Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}
RSpec.configure do |config|
# Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
config.fixture_path = "#{::Rails.root}/spec/fixtures"
# If you're not using ActiveRecord, or you'd prefer not to run each of your
# examples within a transaction, remove the following line or assign false
# instead of true.
config.use_transactional_fixtures = true
# If true, the base class of anonymous controllers will be inferred
# automatically. This will be the default behavior in future versions of
# rspec-rails.
config.infer_base_class_for_anonymous_controllers = false
# Run specs in random order to surface order dependencies. If you find an
# order dependency and want to debug it, you can fix the order by providing
# the seed, which is printed after each run.
# --seed 1234
config.order = "random"
# Include route helpers
config.include Rails.application.routes.url_helpers
#
# Take the FactoryGirl out of FactoryGirl.create
config.include FactoryGirl::Syntax::Methods
end
I'm using Rails 4.0.0.rc1, Ruby 1.9.3, FactoryGirl and rspec-rails 2.13.0 Thanks for any help.
It turns out that Rails 4 is supported starting in rspec-rails 2.13.1 - I was using 2.13.0. After upgrading, the specs took place within a transaction like they were supposed to.
Thanks to everyone who took the time to post help.
I believe the problem is the way you have your test written and less to due with config.use_transactional_fixtures = true. Focus on the bottom of the error that says (compared using ==)
Try to use the expecting change rspec syntax instead
Change this:
click_button 'Create Artwork'
page.should have_text 'Successfully created'
Artwork.count.should eq 1
To this:
expect { click_button 'Create Artwork' }.to change { Artwork, :count }.by(1)
page.should have_text 'Successfully created'
Let me know if this helps
You're running a request spec: when you call visit the code under test is run in a server instance (in the same process). In particular this means that it's using a different thread.
As a result the application code ends up using a different database connection, and since transactions are a per connection thing there is no transaction used when your controller inserts records into the database.
There are several ways to address this. One is to abandon rspec's transactional fixtures and use the database_cleaner gem. You can set it up so that controller and model specs use transactions but request specs use truncate to forcibly clear out tables.
Another approach is to try and force both the spec code and the server code to use the same database connection, this eliminating the problem. You can see this approach in this answer. In my experience this works pretty well until you start using a capybara driver such as poltergeist which will run any javascript on the page and your page fires ajax requests.
The approach I've been using is to set the active record connection pool size to 1: there is only 1 connection allowed so everyone will use the same one. You do then have to do some work to ensure that connections are returned to the pool or your spec just hangs.
I wrote up the details a while ago as a blog post, but in a nutshell you need to
call ActiveRecord::Base.clear_active_connections! before calling methods like visit, click and so on
hack config.middleware.insert_before ActiveRecord::ConnectionAdapters::ConnectionManagement so that it clears the connection after each request (by default it doesn't do this in tests).
I have a spec that tests action caching when caching is disabled and when caching is enabled. It seems like the order of test execution affects whether they pass or not.
it "should not cache the index page when we're not caching" do
ActionController::Base.perform_caching = false
HomeController.caches_action :index
Rails.cache.clear
ActionController::Base.cache_store.exist?(:index_cache_path).should be_false
get :index
ActionController::Base.cache_store.exist?(:index_cache_path).should be_false
end
it "should cache the index page when we're caching" do
ActionController::Base.perform_caching = true
HomeController.caches_action :index
Rails.cache.clear
ActionController::Base.cache_store.exist?(:index_cache_path).should be_false
get :index
ActionController::Base.cache_store.exist?(:index_cache_path).should be_true
end
When tests are run in the above order the last test fails because the cache_store does not exist in the last expectation. I'm stumped on why the no caching test is affecting the caching test. Does anyone know what is wrong?
If you have spec_helper.rb random test order turned to true, it makes sense since you're not un-doing your "ActionController::Base.perform_caching = false" setting.
The recommended way to write caching tests is to do a before(:each) and after(:each) setting the caching settings on and off.
Since you're testing these settings, if you turn it on, remember to turn it off before the end of the test, and vice-versa. Your tests will be more atomic.
Make sure that you have:
config.action_controller.perform_caching = true
In environment/test.rb.
Else - i noticed super strange thing. Caching worked when I run only requests tests (spring rspec spec/requests describe '..' type: :request), but same tests failed if I run everything with rspec spec.