A project that I am working on has integration tests which actually go out and hit a 3rd party api over the wire... Running these tests takes a very long time. I suggested that we implement something like VCR so that the data the tests interact with can be captured as fixtures, and improve reliability and speed of these tests.
I don't have access to the codebase at this moment, but I believe the tests were doing something like this:
before do
login_some_user
end
after do
make_web_request_to_clear_items_in_cart
end
it "adds some items to the user's cart" do
make_web_request_to_add_item_to_a_cart
end
So basically the before block was making a web request, the example was making a totally different request, and an after block (which I know is not ideal to use) made a 3rd request to clean up records that had been created by the example.
I setup an around block in spec_helper that captures web requests and stores them named after the example. However, after running the tests repeatedly I found that they became extremely flakey, sometimes passing, sometimes not.. I tried wrapping the before and after blocks in a separate VCR.use_cassette block call, but it made no difference.
I am wondering if I am missing something, and if there's a way to handle multiple requests like this or what?
VCR can handle multiple requests. From you pseudo-code, you would have the following:
it "adds some items to the user's cart" do
VCR.use_cassette "your_path/cassette_name" do
login_some_user
make_web_request_to_add_item_to_a_cart
make_web_request_to_clear_items_in_cart
end
end
The cassette your_path/cassette_name would contain all 3 web requests.
Your issue likely comes from using before and after blocks. See Myron Marston explanation of how VCR handles the before(:all) hook.
Related
I'm finding Rails integration tests relevant for testing flows and I have some questions about the industry standard on replacing controller test (deprecated in rails 5) with integration tests.
Usually we have tiny controllers where we get the parameters, call the right collaborator and prepare the response and it is easy to test it by mocking the collaborator directly on the controller object.
I am concerned about the overhead of migrating every controller test to integration test that persist the db. What are the standards for this case?
Whats the standard when testing just one route/action and not a complete flow?
How can we replace this?:
#controller.stubs(:authenticate).returns(true)
Integration tests are intended to mimic a real user. They're meant to test the entire application in their entirety.
Opinion varies on what this means. To me, it means you should avoid stubbing/mocking completely. Not a single thing stubbed or mocked, everything executed in full. This means that every integration test I write goes through the actual authentication process of typing in a username and password. Some of the steps are redundant, yes.
Integrations tests are slower all around than unit/controller tests. Cutting out the authentication steps likely won't save you enough time to make a difference in the long run (no pun intended).
We have a very large application that we are currently implementing rspec feature tests. It is a rails application with a knockout.js front-end framework. We have a lot of trouble with wait_for_ajax. We constantly have tests fail because of ajax, and our developers sometimes have to put wait_for_ajax in sometimes three or four times.
This can't be the correct way to do this. What is the correct way to wait for ajax calls in rspec tests?
The hard truth is that javascript tests in capybara are painful and slow.
The only way we can determining if an ajax request if the ajax requests are finished is though hacks like this:
module JavascriptTestHelpers
def wait_for_ajax
Timeout.timeout(Capybara.default_wait_time) do
loop until finished_all_ajax_requests?
end
end
def finished_all_ajax_requests?
page.evaluate_script('jQuery.active').zero?
end
end
I would be really happy if someone proves me wrong. Its almost impossible to get a handle on a specific ajax request unless you do a crazy hack like assigning the promises to the global object. In general this seems to be problematic no matter what the language when automating web browsers.
Running tests in parallel can help a bit with the slowness.
Thoughbot has a really good blog post on some of the common gotchas of capybara JS test which can cause "flapping" tests.
I think that for client heavy applications a javascript test suite in Mocha, Jasmine or (shudder) QUnit is a necessary compliment.
I have a rails application that uses Recurly for its transactions. I am trying to write automated tests for some of the helper functions that I have written.
A super simple example of a function...
def status_for_display
transaction.status.capitalize
end
In order to test these functions, I need to have a Recurly::Account object as well as associated Recurly::Transaction objects.
I have tried going the route of using Recurly::Account.create & Recurly::Transaction.create but I cannot seem to get the transactions to match up with the account.
I am also wondering if it doesn't just make better sense to use the VCR gem to make this happen. In which case, how would I go about doing that? I've never really managed to get VCR setup properly.
VCR is, by in large, plug and play. Once you have it configured and enabled, it'll intercept all HTTP requests and try to play back the data from a cassette. The problem with VCR, though, is that it is request data specific. In order for it to work right, you need to ensure that your tests are always sending the exact same request params to Recurly. You can work around this by having it skip certain things, but it's generally a pain.
The other option is to just use something like Webmock directly and house your own "known responses" for your Recurly calls, but then it's up to you to ensure that you responses stay in sync with the API.
In the end, I'd probably recommend going the VCR route but structuring your tests such that you have known good and bad test scenarios so you can get the real benefits of the cassettes.
Rspec noob here, just trying to improve my test coverage.
One very basic yet important question I have is just: What kinds of tests go where?
Model tests are straight forward. I just need to test the functionality of the models methods and validations. View tests seem simple. That would just be testing that each view renders the desired data.
What confuses me is what exactly goes in my Request Specs. Most of my rails experience is from following Michael Hartle's Rails Tutorial. His Request specs seem to be based around a series of actions that the user could take in the application. But he also includes test which seem like they should be in the View Specs that I am considering moving elsewhere.
If someone could help me understand what kinds of tests go in Request, that would be helpful.
From the RSpec docs:
Request specs provide a thin wrapper around Rails' integration tests, and are
designed to drive behavior through the full stack, including routing
(provided by Rails) and without stubbing (that's up to you).
With request specs, you can:
specify a single request
specify multiple requests across multiple controllers
specify multiple requests across multiple sessions
Check the rails documentation on integration tests for more information.
From Rails' docs on integration tests:
Integration tests are used to test the interaction among any number of controllers. They are generally used to test important work flows within your application.
If your test has to do with how a single view is rendered (which should be completely decoupled from any actual HTTP request), then it's probably better as a view test. If it has to do with multiple views or multiple requests, then an integration test is probably more appropriate.
tl;dr: Jump to the last paragraph
Recently I've been trying to use RSpec's request specs to do some more targeted testing.
This is how my testing mostly looks:
general cucumber feature specification, i.e. user goes to a post with comment, upvotes on a comment and the author gets points
model specs for when the model actually has some functinality, i.e. User#upvote(comment)
controller specs where I stub most of the things and just try to make sure the code goes the way I expect
view specs for when there is something complex in the view, such as rendering a upvote link only when the user didn't already upvote, and these are stubbed as well
The problem is when I have some specific scenario which causes a bug and everything seems to work in the model/view layer where I am unable to reproduce it.
That forces me to write an integration test, which I can also do in cucumber. The problem arises once I am able to actually reproduce it, and I need to figure out why is it happening. This usually means playing around in tests, changing different things and seeing what happens.
For example create a comment that is owned by the user who is trying to upvote, try to vote with an expired session etc. However these are really huge pain to write in Cucumber, because of the need to write a scenario and then specify each step.
At this point, I prefer to write a request spec, because it is more low level and allows me to directly do stuff. The problem is, that I'm not really sure how to properly write a request spec, or what are the rules.
A simple example here is:
visit login_path
fill_in "Username", :with => user.username
fill_in "Password", :with => user.password
click_button "Log in"
vs
post sessions_path(:username => user.username, :password => user.password)
or even something more low level like
session[:user_id] = user.id # this actually doesn't work, but the idea is there
Both of these examples achieve the same thing, they'll log a user in. I know that the answer to which one to pick is based on what I need to test, but that doesn't answer the correct, conventional way to do this.
I've been trying to find something about request specs, but they're not really described anywhere. The RSpec book doesn't cover them, the RSpec documentation doesn't say anything either.
What is a correct way to write request specs? When should I use capybara and when just the Rails' #get and #post methods instead of clicking buttons and visiting paths?
For requests spec I believe the convention is to stick to testing user behaviour and interface interactions, which would mean loading the page, filling in the form etc. A website user cant set the session or interact with variables directly so neither should your request specs.
I've often been tempted to skip page loads and form interactions by posting or setting variables in request specs (for speeds sake, especially heavy ajax specs) but it really does break the purpose of a request spec.
As the comments mentioned, you should test the specific controller / view behaviour in the other spec types.
Sermon first....
I think the natural progression of YOU the test writer goes:
Controller
Model
Requests
a mix of Request, Controller and Model specs.
I know that I started looking at the controller first, cause it was easier to grasp.
You then get into Model specs, for non-happy path things...
Then you realize rspec doesn't actually render the view, so you are starting to see dumb errors in Airbrake, so you say, shoot... I need to test the views, and the workflow. Hence Request specs.
Lastly, you get older and realize, all 3 are important and should be used sparingly, but accordingly. I'm just at step 4 now... too many request specs, and you are slogging 5 mins for the whole suite on a medium sized app. sucks.
To answer your question:
I test workflow and views i need to be "seen" (with page.should or any tricky JS (jquery ui selectors for instance) with capybara in a request spec.
If i just need to make sure the controller variables are instantiated or to do something quick with a post for a non-happy path , out of workflow thing... I use the controller. E.g., POST to IPN Controller in paypal...
You'd be surprised how much this covers. This leaves you to test the models for all the whacked out stuff you need for total all the edge cases.
Honestly though, I'd say use Fixtures and Test Unit for integration tests... still like them better, faster... stronger... etc.