We have a few feature specs that occasionally fail that a rerun fixes. I have a lot of experience testing around Capybara timing, but in this particular case I'm not actually sure how to fix it. Trying to web search for this has been surprisingly ineffective.
Here's the code we have that fails:
expect {
click_on 'Save'
}.to change { Report.count }.from(1).to(2)
There's a couple of different ways I can think to do this:
expect(Report.count).to eq 1
click_on 'Save'
expect(Report.count).to eq 2
Or perhaps:
expect(Report.count).to eq 1
click_on 'Save'
expect(page).to have_something
expect(Report.count).to eq 2
But given that these tests fail so randomly and infrequently, it's difficult to test that we're doing it right.
What is the correct way to verify a count changed based on a click in Capybara?
Firstly, direct DB checks in feature/system tests (which I assume is what you're writing since you're using Capybara is generally a bad code smell since you should generally be verifying things through visual changes on the page (they make more sense for request/controller tests).
If you insist on doing direct DB feature tests you still need to use visual checks to synchronize those checks. This is because actions like click_on have no knowledge of any further browser action those clicks initiate and can just return immediately after clicking. Since the count won't actually be changed until after a request is processed you need to delay until that request is completed (usually done by waiting for a visible page change that indicates it has completed). Therefore the correct way would be
expect(Report.count).to eq 1
click_on 'Save'
expect(page).to have_something # wait for something that indicates the request triggered by click_on has completed
expect(Report.count).to eq 2
which could also be written as
expect {
click_on 'Save'
expect(page).to have_something
}.to change { Report.count }.from(1).to(2)
or more flexibly as
expect {
click_on 'Save'
expect(page).to have_something
}.to change { Report.count }.by(1)
Related
So I am writing an acceptance test using capybara. The scenario was to connect our newsletter system to external mail service.
We will get redirected to our external service page to request access to the external mail service. And we will be redirected back to our system page when succeed.
When "I grant authorization" do
fill_in "username", :with => "myapp"
fill_in "password", :with => "secret"
click_button "Allow Access"
end
Then "I should see 'Connection Established'" do
page.should have_content "Connection Established"
end
And "I should see that I'm connected to Sample External Service" do
page.should have_content "Connection Established: Sample External Service"
page.should have_content "Deactivate Sample External Service syncing"
end
But if I am not using sleep before the page.should have_content "Connection Established". The spec will fail. From what I know, using sleep is not the best practice, because it will make our test run slow.
How to make it waiting until it got redirected back to our system
There are 3 ways to adjust the maximum amount of time Capybaras methods will wait for their expectations to be true/elements to exist
Capybara.default_max_wait_time = <seconds> - This is the global setting which should be set high enough for the vast majority of your method calls
Capybara.using_wait_time(<seconds>) do ... end - This temporarily changes default_max_wait_time inside the block and then returns it to its original setting when done. This makes sense when you have a number of methods you want to call with a new wait time, or you need to call a helper method with the wait time set to a different value.
:wait option - All Capybara finder and expectation methods accept a :wait option which will change the maximum wait time for that method call. This makes sense to use when you have a specific case that requires a bit more waiting than normal
# This will wait up to 10 seconds for the content to exist in the page
page.should have_content "Connection Established: Sample External Service", wait: 10
Note: In the future when posting questions it is generally helpful if you provide the full exact error message you get as part of your question.
For page transitions, I like to wait for the URL to change first, and then wait for content on the page. It gives you a more specific error if the redirect fails, and it naturally splits the long wait into two chunks. With two opportunities to hit default_max_wait_time, the timeout is potentially doubled without actually changing it. If it's still not enough, you can always pass a custom timeout into have_current_path with the wait: parameter.
expect(page).to have_current_path("externalservice.com")
expect(page).to have_content("Connection Established")
There might be a page.should equivalent, but I think the official guidance is to move to the expect syntax, since the should syntax was deprecated.
You can use capybara_watcher gem, it is an elegant way of waiting for the pege to have a change in its content.
Check it out on RubyGems.org
Example:
wait_until_content_has "Connection Established" do |text|
page.should have_content text
end
The perks of using this is that the sleep time is the actual time the page takes to have a change and you can configure it to exit after the second you choose if the page didn't change.
My solution to similar problems:
module WaitUntil
def wait_until(max_wait_time = Capybara.default_max_wait_time)
Timeout.timeout(max_wait_time) do
sleep(0.2) until yield
end
end
end
Using:
wait_until { page.has_css?('li', text: 'test', visible: true) }
You can use Capybara.using_wait_time(seconds) to temporarily change the value of Capybara.default_max_wait_time for special cases:
Capybara.using_wait_time(10) do
page.should have_content "Connection Established"
end
I have a scenario test:
scenario 'displays the form' do
# Build the setup for user coming from an external site.
expect(page).to have_selector('#braintree-dropin-frame', count: 1)
# User reloads the page
expect(page).to have_selector('#braintree-dropin-frame', count: 1)
# User visits page from within the website.
expect(page).to have_selector('#braintree-dropin-frame', count: 1)
end
First off, is this the proper usage of a scenario test? I'm essentially testing the same thing but in different scenarios. I feel like this should really be three separate scenario tests inside a context block. Am I misusing scenario?
If this isn't one user flow then they should be separate features/scenarios. You also need to be careful when setting expectations for the same selector multiple times in a test that you've checked it has gone away between (if you are checking it reappears again), otherwise you can end up with tests passing when they shouldn't due to timing and asynchronous behavior issues.
Additionally - since you're checking for a CSS selector you probably want to be using have_css rather than have_selector since it reads nicer and will mean the tests keep working if the default selector type is ever changed from :css
I have a small trouble with my tests suite: when I run spec which the checking ajax action on the page, sometimes I get random error
Failure/Error: expect(page).to have_content 'DHH'
This error shows very rarely (about 1/100), but this very confused me. I decided this a 'race condition' cause, and I add this config in my spec/rails_helper.rb
Capybara.default_max_wait_time = 10
but this is don't help for me, and I decided add timestamps
it 'adds new DHH', js: true do
find('#modal_new_dhh').click
fill_in('name', with: 'DHH')
p 'click button'
p Time.now.strftime('%H:%M:%S.%L')
click_button('Submit')
p 'checking content'
p Time.now.strftime('%H:%M:%S.%L')
expect(page).to have_content 'DHH'
p 'after checking content'
p Time.now.strftime('%H:%M:%S.%L')
end
and see that
"click button"
"17:34:43.083"
"before checking content"
"17:34:43.127"
"after checking content"
"17:34:43.213"
why Capybara don't wait after click button?
sorry for my bad English
The wait in your example occurs in the have_content matcher. Where you're outputting times from will never show a delay because click_button has nothing to wait for, it just clicks a button and moves on (because it has no idea what it would wait for, clicking a button could do anything), however the have_content matcher will wait up to Capybara.default_max_wait_time for the content to appear.
Note your find, 'fill_in' and click_button calls also wait for the relevant elements to appear BEFORE performing their actions
As you said this is a race condition. As to why it happens, I can't really say, the only reason I could think of would be that there is a difference between a user experience and an automated testing, because computers are very fast.
I had faces the same challenge sometime ago and I found this tutorial which showed me a way to go about resolving these kind of issues. I hope it would be useful to you too.
What is considered a solid spec?
This is what I find to be very abstract about testing. I'd be interested in the answer for this on models, controllers and whatever else can be tested. It would be cool to have a spec for a spec, you know what I mean?
A model spec should (in order of priority and relevance):
Test all methods?
Test errors array?
Test CRUD (and how)?
What else?
A controller / view spec should (in order of priority / relevance):
Fill in the blank...
?
Would be great to expand this list of what a spec should and shouldn't contain.
I'd also like to compile a list of tricks and suggestions as well. For example:
The keyword "should" is sorta redundant.
Example:
this:
it "should be invalid without a firstname"
would be better as:
it "is invalid without a firstname"
Yet another trick, use expect instead of lambda for readability:
lambda { ... }.should be_valid
is more readable as:
expect { ... }.should be_valid
I am compiling a list of helpful articles on getting started and will share those in this post as they come along. Here are some that I'm finding particularly helpful as of now. (Feel free to post yours and I'll tack it on if it seems helpful).
http://everydayrails.com/2012/03/19/testing-series-rspec-models-factory-girl.html
http://nelvindriz.tumblr.com/post/835494714/rspec-best-practices
It would be great to have a list of projects where tests are implemented well. Since rspec is so readable (at least that's what everybody says), it would be great to get a list of links to projects that have great specs to read.
"See the Mongoid specs for an example of good specs." -#yfeldblum (see answer below)
Online you'll find a lot of articles describing unrealistic scenarios on how to test basic stuff, but beyond that you're sorta on your own. If I were to write an article on this topic I would just link to my tests (on github for example), then thoroughly annotate one or a few of those specs... this seems like the best way to write an article on rspec, in my opinion. I'd do it myself, but I'm not quite there yet.
If you vote to close this, that's fine, just try to leave a comment or suggestion on where you think this post would belong. Thanks!
This is actually a good question because when I started out with test cases, I wasn't sure what is considered a good test case. Here are a few things which you can follow. This list is not mine; but compiled from a few sources plus some of my additions.
Describe methods
While describing methods, it is a good practice actually describe your method like: describe "#admin?" etc. "." is a prefix for class method and "#" is a prefix for instance methods.
One assert per test case
Make sure that you have just one assertion per test case. This makes sure that your test cases are clean and easy to understand; which is the point of test cases, isn't it? :)
Avoid saving data to db
You can dynamically build objects to and avoid saving data to db. Although you can clean up the db before each test case, "not saving" will speed up test cases in a big way.
#user.build(:something) instead of #user.create(:something)
Edge and Invalid cases
This is not specific to Rspec but it is important to make sure edge cases are covered in testing. This helps greatly later on when your project grows and it gets easy to maintain.
contexting
I, personally, like this a lot in Rspec and I in fact overuse contexts a bit. Using contexts with conditions helps in compartmentalizing your test cases. Here's an example:
# Avoid
it "should have 200 status code if user is logged in" do
response.should respond_with 200
end
it "should have 401 status code if user is not logged in" do
response.should respond_with 401
end
# Use
context "when user is logged in" do
it { should respond_with 200 }
end
context "when user is logged out" do
it { should respond_with 401 }
end
Using subject
When you have lots of test cases which are related to the same thing, you can use subject() to make sure you don't repeat yourself.
# Avoid
it { assigns(:user).should be_valid }
it { assigns(:user).should_not be_dumb }
it { assigns(:user).should be_awesome }
# Use
subject { assigns("user") }
it { should be_valid }
it { should_not be_dumb }
it { should be_awesome }
Here are a few things that I try to follow when I write test cases. I'm sure there are a lot more things which can improve Rspec test cases. But this should be enough to get started and write awesome test cases.
See the Mongoid specs for an example of good specs.
Have exactly one assertion per example. Do not assert two things in the same example. An example is the block passed to it, its, or specify.
I'm following a tutorial to learn Ruby on Rails, and it's teaching Rspec as part of test-driven development. The flow here is to write a test that fails, write code to pass the test, and then pass the test. The rationale seems to be that by doing this—by starting with a failing test—you can be pretty darn sure that your code does what you expect. So I suppose a good spec is one that ensures that your code does what it's supposed to. As of yet I haven't gleaned any rules of thumb from the tutorial like the other posters have written, though.
Here's the link: http://ruby.railstutorial.org/
When you are writing your controller tests you want to cover with tests all your actions in your controller.
Example how tests for controller actions should look like:
describe "Stories" do
describe "GET stories#index" do
context "when the user is an admin" do
it "should list titles of all stories"
end
context "when the user is not an admin" do
it "should list titles of users own stories" do
end
end
describe "GET stories#show" do
it "should render stories#show template" do
end
end
describe "GET stories#new" do
it "should render stories#new template" do
end
end
describe "POST stories#create" do
context "with valid attributes" do
it "should save the new story in the database"
it "should redirect to the stories#index page"
end
context "with invalid attributes" do
it "should not save the new story in the database"
it "should render stories#new template"
end
end
describe "DELETE stories#delete" do
it "should delete the story from the database"
it "should redirect to the stories#index page"
end
end
You can find out more about controller tests here.
Artists cannot rate their own artworks
it "should display error if voting on own artwork", :js => true do
sign_in
visit "/upcoming"
click_link "like_post_1"
page.should have_content("Can't vote on your own artwork")
end
This was passing just fine.
However, I can't click on like_post_1 anymore because I added a feature to prevent voting links from appearing next to your own artworks.
Does this mean I no longer need test coverage for this scenario because it's extremely rare that someone can click on a voting link for their own artwork? Or should still have coverage to test the ajax response, because it's not tested anywhere else and it's possible for some stale page of links to somehow exist in a tabbed browser window. If so... how do I test it if I cannot call click_link?
I could try to create a POST request to create the vote, but capybara doesn't support posts, and I can't test the ajax response that way...
Or is there a way to simulate tabbed browsing in capybara?
Suggestions?
You can use CSS display:none or visibility:hidden for the own artwork instead of eliminating the link from the DOM. You might have to set
Capybara.ignore_hidden_elements = false
Another way is giving up Capybara and putting them into the controller/model spec. Controller/model spec might be a better place for the extremely rare case or safeguarding your app case.
Please try this:
Use sleep function.
sleep 10 use to wait process upto 10 seconds..
it "should display error if voting on own artwork", :js => true do
sign_in
visit "/upcoming"
click_link "like_post_1"
sleep 10
page.should have_content("Can't vote on your own artwork")
end