Facts:
Running my entire suite of specs will result in 21 consistent errors out of 610 specs.
If I run any individual spec file (for instance: messages_controller_spec.rb), they will all pass.
If I run any of the failed specs individually, they will each pass.
These errors are mostly ActionMailer failures, but some are otherwise.
One confusing aspect is that some of the specs fail because there was an extra row in the database than expected, while others fail because there was one less row than expected. That is, if it were a cleaning or cache issue, it seems like it should be consistently one more or one less.
I’m currently running Rails 4.1.1, Ruby 2.0.0p451, Rspec 2.14.8, Sidekiq via inline!
Gemfile(for test)
group :development, :test do
gem 'better_errors'
gem 'binding_of_caller'
gem 'faker'
gem 'guard-rspec'
gem 'pry'
gem 'rspec-rails'
gem 'spork-rails'
gem 'sqlite3'
gem 'thin'
end
group :test do
gem 'capybara'
gem 'capybara-email'
gem 'capybara-webkit'
gem 'database_cleaner'
gem 'fabrication'
gem 'launchy'
gem 'selenium-webdriver'
gem 'shoulda-matchers'
gem 'webmock'
gem 'vcr'
end
Please note, I’ve checked out about a dozen similar questions that did not work to fix this. So, to clarify:
I’m not using ARGV
I’m not using before(:all) - I use before(:each).
I have attempted Rails.cache.clear before both the :suite & the :each spec.
I have database cleaner set to :truncation for all cleaning (slower, but better results than :transaction - have had issues with reloading updated values using :transaction)
For convenience sake, maybe some examples will help:
scheduler_spec.rb (showing test that fails when whole suite is run)
require 'spec_helper'
require 'rake'
require 'sidekiq/testing'
Sidekiq::Testing.inline!
describe "scheduler", :vcr do
describe ":wipe_abandoned_images" do
let!(:abandoned_old_image) { Fabricate(:image) }
let!(:abandoned_young_image) { Fabricate(:image) }
let!(:adopted_image) { Fabricate(:image) }
let(:run_cleaner) do
Rake::Task[:wipe_abandoned_images].reenable
Rake.application.invoke_task :wipe_abandoned_images
end
before do
abandoned_young_image.update_columns(listing_id: nil, updated_at: 6.days.ago)
abandoned_old_image.update_columns( listing_id: nil, updated_at: 9.days.ago)
Rake.application.rake_require 'tasks/scheduler'
Rake::Task.define_task(:environment) #Stub env. Rspec runs the App, so dont want Rake to run it again.
end
context "for claimed images" do
it "leaves the image" do
adopted_image_id = adopted_image.id
run_cleaner
expect(Image.all.count ).to eq(2)
expect(Image.find(adopted_image_id) ).to be_present
end
end
end
end
Note that I'm using Sidekiq's inline! testing configuration on prior apps with good success & without this issue.
forgot_passwords_controller_spec.rb (showing test that fails when whole suite is run)
require 'spec_helper'
require 'sidekiq/testing'
Sidekiq::Testing.inline!
describe ForgotPasswordsController do
let!(:jen) { Fabricate(:user, email: 'jen#example.com') }
describe "POST create" do
context "with valid email provided" do
before { post :create, email: 'jen#example.com' }
after do
ActionMailer::Base.deliveries.clear
Sidekiq::Worker.clear_all
end
it 'sends the reset email to the users provided email' do
expect(ActionMailer::Base.deliveries.count).to eq(1)
end
end
end
end
Here's what happens when I run the specs in various ways:
ForgotPasswordsController Specs pass via RubyTest plugin in SublimeText2
2014-08-28T03:42:43Z 32968 TID-ov5g65p44 INFO: Sidekiq client with
redis options {} ........... Finished in 0.95479 seconds 11 examples,
0 failures Randomized with seed 40226 [Finished in 5.8s]
Scheduler Tests pass via RubyTest in SublimeText2
.Rationing out invitations to users... done. .Rationing out
invitations to users... done. ..Sweeping the server for abandoned
images... 2014-08-28T01:49:02Z 32426 TID-owjt9ggh8 INFO: Sidekiq
client with redis options {} done. .Sweeping the server for abandoned
images... done. .Sweeping the server for abandoned images... done.
.
Finished in 1.52 seconds 7 examples, 0 failures Randomized with seed
37996 [Finished in 8.6s]
Tests pass via rspec in Console
$ rspec ./spec/lib/tasks/scheduler_spec.rb
.Rationing out invitations to users... done. .Rationing out
invitations to users... done. ..Sweeping the server for abandoned
images... 2014-08-28T02:14:43Z 32456 TID-ouiui9g8c INFO: Sidekiq
client with redis options {} done. .Sweeping the server for abandoned
images... done. .Sweeping the server for abandoned images... done.
.
Finished in 1.32 seconds 7 examples, 0 failures Randomized with seed
19172
Tests fail when run entire suite of Rspec
These failures will pass if run individually or as the spec file they are from.
$ rspec
Finished in 49.71 seconds 610 examples, 21 failures, 10 pending
Failed examples:
Sometimes it has an extra value
13) scheduler :wipe_abandoned_images for abandoned images under 1
week old leaves the image
Failure/Error: expect(Image.all.count ).to eq(2)
expected: 2
got: 3
(compared using ==)
# ./spec/lib/tasks/scheduler_spec.rb:78:in `block (4 levels) in <top (required)>'
Sometimes it loses or doesn’t load a value
18) ForgotPasswordsController POST create with valid email provided
sends the reset email to the users provided email
Failure/Error: expect(ActionMailer::Base.deliveries.count).to eq(1)
expected: 1
got: 0
(compared using ==)
# ./spec/controllers/forgot_passwords_controller_spec.rb:22:in `block (4 levels) in <top (required)>'
Here's a list of the failures
rspec ./spec/controllers/messages_controller_spec.rb:161 # MessagesController POST create message about listing to user from guest with valid information with EXISTING, UN-confirmed guest with EXPIRED token sends another confirmation email with link to the guest
rspec ./spec/controllers/messages_controller_spec.rb:114 # MessagesController POST create message about listing to user from guest with valid information with NEW, UN-confirmed, and valid guest email sends an invitation for the guest to be put on safe-email list
rspec ./spec/controllers/invitations_controller_spec.rb:30 # InvitationsController POST create with valid email & available invitations sends an email
rspec ./spec/controllers/invitations_controller_spec.rb:33 # InvitationsController POST create with valid email & available invitations sends an email to the recipient_email address
rspec ./spec/controllers/users_controller_spec.rb:161 # UsersController POST create with invitation token in params with valid token & input confirmation email sending sends the email to the registering user
rspec ./spec/controllers/users_controller_spec.rb:158 # UsersController POST create with invitation token in params with valid token & input confirmation email sending sends the email
rspec ./spec/controllers/users_controller_spec.rb:164 # UsersController POST create with invitation token in params with valid token & input confirmation email sending sends an email with a confirmation link in the body
rspec ./spec/controllers/users_controller_spec.rb:354 # UsersController GET confirm_with_token with valid token has a welcome message in the email
rspec ./spec/controllers/users_controller_spec.rb:348 # UsersController GET confirm_with_token with valid token sends a welcome email
rspec ./spec/controllers/users_controller_spec.rb:351 # UsersController GET confirm_with_token with valid token sends the welcome email to the user
rspec ./spec/controllers/searches_controller_spec.rb:19 # SearchesController GET search GET search with specific category selected returns the matching OR partial-matching table row objects
rspec ./spec/controllers/searches_controller_spec.rb:22 # SearchesController GET search GET search with specific category selected only returns values from the selected category
rspec ./spec/lib/tasks/scheduler_spec.rb:75 # scheduler :wipe_abandoned_images for abandoned images under 1 week old leaves the image
rspec ./spec/lib/tasks/scheduler_spec.rb:68 # scheduler :wipe_abandoned_images for abandoned images over 1 week old deletes the images
rspec ./spec/lib/tasks/scheduler_spec.rb:84 # scheduler :wipe_abandoned_images for claimed images leaves the image
rspec ./spec/controllers/forgot_passwords_controller_spec.rb:24 # ForgotPasswordsController POST create with valid email provided sets the email subject to notify the user of the reset link
rspec ./spec/controllers/forgot_passwords_controller_spec.rb:27 # ForgotPasswordsController POST create with valid email provided sends the link with token in the body of the email
rspec ./spec/controllers/forgot_passwords_controller_spec.rb:21 # ForgotPasswordsController POST create with valid email provided sends the reset email to the users provided email
rspec ./spec/controllers/reset_passwords_controller_spec.rb:70 # ResetPasswordsController POST create with a valid token sets the email subject to notify the user of the reset password
rspec ./spec/controllers/reset_passwords_controller_spec.rb:67 # ResetPasswordsController POST create with a valid token sends a confirmation email to the user that their password has been changed
rspec ./spec/controllers/reset_passwords_controller_spec.rb:73 # ResetPasswordsController POST create with a valid token sends the link with token in the body of the email
I can't explain what is really causing the error. But it must due to setting the Sidekiq test mode globally. Remove the Sidekiq setting from the head section of the specs and try the following:
before do
Sidekiq::Testing.inline! do
post :create, email: 'jen#example.com'
end
end
after do
ActionMailer::Base.deliveries.clear
Sidekiq::Worker.clear_all
end
it 'sends the reset email to the users provided email' do
expect(ActionMailer::Base.deliveries.count).to eq(1)
end
Related
I am new in rails testing and I have written an integration test for user signup. Test is working fine but it is not inserting record in the database.
Here is the code
require 'test_helper'
class SignupTest < ActionDispatch::IntegrationTest
test "user signup" do
visit new_user_registration_path
fill_in "user_email", with: "abc#gmail.com"
fill_in "user_password", with: "password"
fill_in "user_password_confirmation", with: "password"
click_button "Sign up"
assert_text "Welcome! You have signed up successfully."
end
end
This is the command that I am using to run the test
rake test:integration
When I run the test the results are
Run options: --seed 62721
# Running:
.
Finished in 3.116669s, 0.3209 runs/s, 0.3209 assertions/s.
1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
I have also checked the logs but nothing in the log.
This is my gemlist for testing
group :test do
gem 'capybara'
gem 'capybara-webkit'
gem 'vcr'
gem 'webmock'
gem 'launchy'
end
Test database is cleaned up after each test run so you will not be able to see any records after you run your test suite (depending on the cleaning method you won't be able to see anything in the DB even during the test if you use different connection to the DB).
If you want to test that user was saved, you need to do it inside the integration test. For example
# ...
assert_text "Welcome! You have signed up successfully."
assert(User.where(email: "abc#gmail.com").exists?)
Or event better - write a unit test that checks it.
Related answer
I am using Machinist to create my test users which I want to log in before each test which I am running with Capybara:
include Capybara::DSL
include ActionController::UrlWriter
before do
SslRequirement.disable_ssl_check = true
user = User.make
visit new_user_session_path
fill_in('username', with: user.email)
fill_in('password', with: '12345')
click_button('submit')
end
it "responds as expected" do
# ...
end
Now when the user is created with User.make I can access it through User.all when it fact it's not persisted in the database yet, this is because
ActiveRecord::Base.connection.open_transactions == 1
I can fix/hack this by doing a
ActiveRecord::Base.connection.commit_db_transaction
after User.make, but I'd rather fix this properly. Is this a Capybara configuration which I am missing? Disclaimer: Using Rails 2.3 and RSpec 1.3
You don't show which driver you're using with Capybara, but generally you shouldn't be using transactions for testing when using Capybara (yes there are potential workarounds to allow it but they all have side-effects). See transactions and database setup and then look into the database_cleaner gem with truncation - here's a blog post about it that should be good with the age of gems you're using - http://devblog.avdi.org/2012/08/31/configuring-database_cleaner-with-rails-rspec-capybara-and-selenium/
I'm trying to use capybara on a ruby on rails application to do some content testing, as well I'm using the devise gem to implement user authentication. I'm having trouble logging into my application to perform my test cases.
Initially, my scenario was as follows:
scenario "User arrives at main page" do
visit "purchase_orders#index"
page.should have_content("All")
# some more tests and checks
end
Where purchase_orders#index is the authenticated root, where a user's purchase orders are shown.
But when I was running the tests, I was getting the following error :
expected to find text "All" in "Log in to manage your orders * Email * Password Forgot your password? Remember me Sign up • Didn't receive confirmation instructions? About Us • Contact Us •
which tells me that its not getting past the log in page. I next tried adding the following to my scenario, before running the tests, to make it log in:
visit "purchase_orders#index"
fill_in('Email', :with => 'username#gmail.com')
fill_in('Password', :with => 'password')
click_button('Log in')
where username and password are actual created accounts, but again it fails and doesn't get past the sign in page. Finally, I tried adding a before(:each) method, as follows, to attempt to sign users in for test cases:
before(:each) do
visit "users/sessions#new"
fill_in('Email', :with => 'nico.dubus17#gmail.com')
fill_in('Password', :with => 'password')
click_button('Log in')
end
which, again, did not work for me. So my question is: What is the best practice and syntax for getting past the sign in page, and into the actual application? I've looked for documentation and answers on this, but haven't found anything.
Thank you!
Found an answer. I installed the warden gem to (gem 'warden') and factory girl gem (gem "factory_girl_rails", "~> 4.0") and ran a bundle install. I then created a user fixture with factory girl as follows in a factory.rb file in the spec folder:
# This is a user factory, to simulate a user object
FactoryGirl.define do
factory :user, class: User do
first_name "John"
last_name "Doe"
email "nico_dubus#hotmail.com"
encrypted_password "password"
end
end
in my rails helper file, I added this line to be able to use FactoryGirl's methods without calling it on the class every time:
config.include FactoryGirl::Syntax::Methods
Afterwards, I added these lines to the top of my capybara test page:
include Warden::Test::Helpers
Warden.test_mode!
Finally, I built a user object to use within the scope of the test:
user = FactoryGirl.build(:user)
And whenever I need to log in, I use warden's log in method
login_as(user, :scope => :user)
And voila!
I've been building my application and I'm now ready to start testing. I have Factory girl defined in seeds.rb but as I'm running tests I've also defined the tests in the usual place /spec/factories.rb for Rspec.
However my first tests fails with the following error.
user_spec.rb
require 'spec_helper'
describe User do
it "should have valid factory" do
FactoryGirl.build(:user1).should be_valid
end
end
Error returned:
Failures:
1) User should have valid factory
Failure/Error: FactoryGirl.build(:user1).should be_valid
NoMethodError:
undefined method `first_name=' for #<User:0x000000061410f8>
# ./spec/models/user_spec.rb:5:in `block (2 levels) in <top (required)>'
Finished in 0.05459 seconds
1 example, 1 failure
Failed examples:
rspec ./spec/models/user_spec.rb:4 # User should have valid factory
Randomized with seed 29084
spec/factories.rb
FactoryGirl.define do
factory :admin1, class: User do
first_name "admin"
last_name "minstrator"
password "admin1234"
profile_name "profilename"
email "admin#admin.com"
password_confirmation "admin1234"
admin true
end
factory :user1, class: User do
first_name "user2"
last_name "man2"
password "user1234"
profile_name "profilename"
email "user2#user.com"
password_confirmation "user1234"
admin false
end
end
It works fine when using the data on development in my seed.rb but now I've started testing using Rspec it has all gone wrong.
What am I doing incorrectly here. I'm not a huge fan of testing at the moment but I need to improve my skill set here as I know it can be extremely useful for web applications.
You help is greatly appreciated, please let me know if you need anymore info.
If your code works in development, but not in test, my guess is that your test copy of the database is not in sync. Try running rake db:test:prepare or rake db:test:clone and run your specs again.
Note:
db:test:clone isn't required in Rails 4.2.0 'WARNING: db:test:clone is deprecated. The Rails test helper now maintains your test schema automatically, see the release notes for details.')
I believe the issue here was that one of my migrations was missing and because it didn't have users first_name in the model there was nothing defined.
A big tip to all newbies, never ever ever delete any of your migrations.
As Homer Simpson quite rightly said "D'oh"
I'm following the rails_admin README. I have setup devise and cancan, only users of group admin can access rails_admin.
Here is the test
test "try to access rails_admin as a non admin user" do
get_via_redirect '/users/auth/facebook'
assert_response :success
assert_match 'Successfully authenticated', flash[:notice]
puts User.all.count
puts User.first.name
puts Group.find(User.first.group_id).name
assert_equal Ability.new(User.first).can?(:access, :dashboard), false
puts session
get '/admin'
assert_response :found
assert_redirected_to '/'
assert_match 'You are not authorized', flash[:alert]
end
When I run 'ruby -Itest test/integration/test.rb' all passes. The output is
1
Facebook User
user
{"session_id"=>"5345e64582b2557d0d02cd2011461467", "warden.user.user.key"=>["User", [2], "$2a$04$B.nVokuCXSWOpZ2Ezf60Cu"], "flash"=>#<ActionDispatch::Flash::FlashHash:0xb49d9a4 #used=#<Set: {:notice}>, #closed=false, #flashes={:notice=>"Successfully authenticated from facebook account."}, #now=nil>}
When I run 'bundle exec rake test:integration' the test fails. The output is
1
Facebook User
user
{"session_id"=>"0005d96c17c75d0843166e5dbb4dcc05", "warden.user.user.key"=>["User", [3], "$2a$04$4f5/I9uSZbMWBdCgDA086O"], "flash"=>#<ActionDispatch::Flash::FlashHash:0xa4c0db0 #used=#<Set: {:notice}>, #closed=false, #flashes={:notice=>"Successfully authenticated from facebook account."}, #now=nil>}
F.
Finished tests in 1.054687s, 3.7926 tests/s, 24.6519 assertions/s.
1) Failure:
test_try_to_access_rails_admin_as_a_non_admin_user(RailsAdminTest):
Expected response to be a <:found>, but was <200>
I also tried "assert_select 'body', 'something'". When running rake test:integration it outputs some html like a standard rails_admin dashborad page. Looks like the user is authorized to access rails_admin.
I'm using rails 3.2.6, devise 2.1.2, cancan 1.6.8, rails_admin 0.0.5
Any ideas? Thanks.
I was having the same issues (single test running but not all at once). It seems this is due to rails_admin not being initalized by default when running test.
What helped was setting
SKIP_RAILS_ADMIN_INITIALIZER=false
before calling rake. So now it's
SKIP_RAILS_ADMIN_INITIALIZER=false rake test
for me.
Unfortunately i do not know how to reinitialize rails_admin from inside the tests (there is a rake task to disable but not to enable) and prefixing rake test every time is annoying. Also the tests do run quite a bit longer if rails_admin is initalized (that's propably why it's turned of by default).
We now use a small test helper for those tests which check rails_admin access. The helper returns immediately and logs a warning if SKIP_RAILS_ADMIN_INITIALIZER is true and thus does not run the tests on normal dev systems. On our continuous integration system SKIP_RAILS_ADMIN_INITIALIZER=false is set and the tests are run.