We have a very large application that we are currently implementing rspec feature tests. It is a rails application with a knockout.js front-end framework. We have a lot of trouble with wait_for_ajax. We constantly have tests fail because of ajax, and our developers sometimes have to put wait_for_ajax in sometimes three or four times.
This can't be the correct way to do this. What is the correct way to wait for ajax calls in rspec tests?
The hard truth is that javascript tests in capybara are painful and slow.
The only way we can determining if an ajax request if the ajax requests are finished is though hacks like this:
module JavascriptTestHelpers
def wait_for_ajax
Timeout.timeout(Capybara.default_wait_time) do
loop until finished_all_ajax_requests?
end
end
def finished_all_ajax_requests?
page.evaluate_script('jQuery.active').zero?
end
end
I would be really happy if someone proves me wrong. Its almost impossible to get a handle on a specific ajax request unless you do a crazy hack like assigning the promises to the global object. In general this seems to be problematic no matter what the language when automating web browsers.
Running tests in parallel can help a bit with the slowness.
Thoughbot has a really good blog post on some of the common gotchas of capybara JS test which can cause "flapping" tests.
I think that for client heavy applications a javascript test suite in Mocha, Jasmine or (shudder) QUnit is a necessary compliment.
Related
Is there anyway to load a page in capybara in a before :all block, and then run each test on the same page?
It seems right now at the end of each test the page is unloaded, so I have to do the setup in a before :each block, which really slows things down.
You can do that by not requiring capybara/rspec and instead implementing you're own before after blocks and RSpec configuration that don't call Capybara.reset_sessions, however that defaults the whole concept of tests being independent, and potentially also makes your tests order dependent which really isn't a good idea.
If your issue is logging in each time you should look into fast tracking the login (for instance test mode in Devise - https://github.com/plataformatec/devise/wiki/How-To:-Test-with-Capybara)
Rspec noob here, just trying to improve my test coverage.
One very basic yet important question I have is just: What kinds of tests go where?
Model tests are straight forward. I just need to test the functionality of the models methods and validations. View tests seem simple. That would just be testing that each view renders the desired data.
What confuses me is what exactly goes in my Request Specs. Most of my rails experience is from following Michael Hartle's Rails Tutorial. His Request specs seem to be based around a series of actions that the user could take in the application. But he also includes test which seem like they should be in the View Specs that I am considering moving elsewhere.
If someone could help me understand what kinds of tests go in Request, that would be helpful.
From the RSpec docs:
Request specs provide a thin wrapper around Rails' integration tests, and are
designed to drive behavior through the full stack, including routing
(provided by Rails) and without stubbing (that's up to you).
With request specs, you can:
specify a single request
specify multiple requests across multiple controllers
specify multiple requests across multiple sessions
Check the rails documentation on integration tests for more information.
From Rails' docs on integration tests:
Integration tests are used to test the interaction among any number of controllers. They are generally used to test important work flows within your application.
If your test has to do with how a single view is rendered (which should be completely decoupled from any actual HTTP request), then it's probably better as a view test. If it has to do with multiple views or multiple requests, then an integration test is probably more appropriate.
A project that I am working on has integration tests which actually go out and hit a 3rd party api over the wire... Running these tests takes a very long time. I suggested that we implement something like VCR so that the data the tests interact with can be captured as fixtures, and improve reliability and speed of these tests.
I don't have access to the codebase at this moment, but I believe the tests were doing something like this:
before do
login_some_user
end
after do
make_web_request_to_clear_items_in_cart
end
it "adds some items to the user's cart" do
make_web_request_to_add_item_to_a_cart
end
So basically the before block was making a web request, the example was making a totally different request, and an after block (which I know is not ideal to use) made a 3rd request to clean up records that had been created by the example.
I setup an around block in spec_helper that captures web requests and stores them named after the example. However, after running the tests repeatedly I found that they became extremely flakey, sometimes passing, sometimes not.. I tried wrapping the before and after blocks in a separate VCR.use_cassette block call, but it made no difference.
I am wondering if I am missing something, and if there's a way to handle multiple requests like this or what?
VCR can handle multiple requests. From you pseudo-code, you would have the following:
it "adds some items to the user's cart" do
VCR.use_cassette "your_path/cassette_name" do
login_some_user
make_web_request_to_add_item_to_a_cart
make_web_request_to_clear_items_in_cart
end
end
The cassette your_path/cassette_name would contain all 3 web requests.
Your issue likely comes from using before and after blocks. See Myron Marston explanation of how VCR handles the before(:all) hook.
I have a Rails 2.3.5 app which is serving a card game. I've been a bit lax in writing tests for it (read: I haven't written any :embarrassed:), and I'd like to make a start now.
From reading other questions, I think I probably want to be using Shoulda extending Test::Unit for unit testing.
I wondered about using Capybara extending RSpec for functional testing, but the majority of users' interaction with the app is via POST, which I understand Capybara doesn't handle.
Since this is a card game, I obviously need the ability to control rand; I also definitely need the framework to handle Javascript.
Is Test::Unit + Shoulda suitable?
Am I correct that Capybara can't handle POST? Can I work around that?
Is there a better option instead of Capybara?
Can these methods handle the randomness involved in a card-game? Presumably, for Test::Unit at least, I can just call srand at some early stage to fix the generator, but I don't know if that's doable for Capybara/RSpec/anything else.
Can anyone point me to resources dealing specifically with setting up Shoulda, Capybara and RSpec for Rails 2.3.5? It's a little hard to tell what's Rails 3 specific and what isn't.
Would I gain anything from adding Watir / Firewatir into the mix?
I realise the above questions are manifold, but they basically boil down to "does this work, or have you any better suggestions?"
If the majority of the users' interactions are via POST, is it via an API (as opposed to filling out forms or something)?
Just about any combination of RSpec/Shoulda/Capybara/Test Unit/Rack::Test could work for you depending on your need. They're all capable. However, these are my recommendations:
If you've decided you want integration testing via HTML pages, use Cucumber and Capybara
If you've decided you want integration testing via HTTP API, use RSpec and Rack::Test
You probably want to fake out randomness.
You probably don't need Watir/Firewatir.
It does look like you can make POST requests via some Capybara drivers:
http://suffix.be/blog/capybara-post-requests
When Rails moved to 3.0, RSpec went to 2.0 so for at least RSpec, you'd want RSpec and RSpec Rails 1.3.2.
By "fake out randomess", I mean redefine srand in your tests so you can predictably run them.
module Kernel
def rand
YourApp.rand
end
end
module MyApp
class << self
attr_accessor :rand
end
end
Then, before you have the user press the button, run a step definition like "When random returns 6", which will set MyApp.rand = 6.
I'm getting started with cucumber, and I have written the following step definition, which basically I copied from the rspec test I already had.
Given /^There is a picture on the screen$/ do
describe PagesController do
describe "GET 'home'" do
it "should be successful" do
get 'home'
response.should be_success
end
end
end
end
Now I'm also quite new to Rails. But this seems a bit long to me, can't I make it any shorter?
What you're doing there is a big antipattern when writing tests with Cucumber. You haven't broken down your step far enough—when you look at the test you've got, you're actually going through at least a couple of different steps, one of which isn't something you want to test. A better test for this would be:
Given I am on the home page
Then I should see a picture
The first step you get for free when you install capybara's web steps, provided you are using the default path helper in features/support/paths.rb. The second one would look like this:
Then "I should see a picture" do
page.should have_selector("img.picture")
end
That step is going to look for an image with a 'picture' class on it—my arbitrary definition of what a picture is in the context of your application.
Notice that I'm not checking the response status here. The idea behind cucumber (even moreso than Rspec) is that you test things from the perspective of the client. Arguably your client may be a client API, so perhaps checking the status code is appropriate, but in general with a web app, you're much more concerned about the UI, and a failing status code will manifest itself in other ways, such as a broken UI. Details like status codes are generally implementation details that shouldn't be tested from a BDD point of view (though they should be covered in your unit tests.)
FWIW, I also disagree with the assertion that the length of the test is what should determine whether you should be using Rspec or Cucumber. See my blog post on the matter: http://collectiveidea.com/blog/archives/2011/04/15/language-matters/
It would strongly recommend not using Cucumber until you're more familiar with Rails/Ruby. Cucumber adds a level of abstraction that can make things very confusing, frustrating and time consuming. Just get very good with RSpec first. Then when you want to do integration tests on your Rails app, use Capybara with RSpec. When you're comfortable using those gems, you can go to Cucumber.
Cucumber is about writing user stories. You then translate those Cucumber steps into lines of RSpec and Capybara.
If you are looking for brevity, I would recommend sticking to RSpec. Cucumber's main advantage is its readability to people who do not write code, while RSpec sacrifices that in order to be shorter. I tried to use Lettuce, a python analogue to Cucumber, on one of my projects and found it to be way too much work to keep up.
So, in short: if you want to write shorter tests, use RSpec. If you want your developer to edit your tests or you are working in a team, use Cucumber.