I would like to run RSpec to test my code both when I'm connected to the web and when I'm not.
Unfortunately there are some tests in the application which are dependent on having a live connection - email sending, Facebook integration etc.
Is there a best-practice way to create an online/offline testing environment or is this bad practice? Judging by how little I can find about this on the web I'm guessing the latter.
Normally in situations like that you would mock the parts of the code that connect outside your application. This allows you to test against the expected results from the service/system you are connecting to. It's also quicker to run tests.
There's a brief tutorial on mocking with rspec here but I'm sure you can find plenty yourself.
For testing that emails get sent there are other approaches if you are sending through ActionMailer. There's a section on that in the rails testing guide.
EDIT (in response to comment):
You could put a method in TestHelper to only run tests when you are online. Something like:
def when_online
if test_remote_connectivity
yield
else
puts "Skipping test offline."
end
end
Then you can call it like:
def test_facebook
when_online do
.....
end
end
Not sure I entirely advocate it but it might do what you want!
You could use webmock inside the tests/specs you don't want connecting to the remote resource.
Related
I have a rake task in my rails application that published jobs to facebook. And then changes some model values. So one way to test the rake task is to invoke the rake task and check values that have been changed.
test 'z' do
# setup some data
Rake::Task['job:publish_to_facebook'].invoke
Rake::Task['job:publish_to_facebook'].reenable
# assert table values that has been changed.
end
But how I can test whether jobs are successfully published on facebook? Is there any better strategy except using capybara and selenium-webdriver ?
Even if i use stubbing and mocking then how can i verify that my jobs are published on facebook?
Most tests should not contact an external API, mainly because it will slow down tests and you might also run into rate limits.
Even if i use stubbing and mocking then how can i verify that my jobs are published on facebook?
The point of stubbing and mocking is precisely not to publish to Facebook. Instead, you would create a class called Facebook (for example) with a method like def post_message(message). This is the app's front door to Facebook, all calls to Facebook go through this class. Then you can use a library like Mocha to overwrite def post_message during testing. You can use it to verify that the application is attempting to post a message, and verify the message itself is correct. It won't actually post the message.
As I mentioned, you do want to make some tests with real calls to Facebook (though not many). These could be in a test like you've shown above, which is an integration test, or it could also be a smaller unit test of the Facebook class I suggested above, which would be a better starting point. For this purpose, you'd want to establish a test account on Facebook. Then your test should clear all messages in the setup and use Facebook's API to verify that the messages were actually posted.
My frontend is an EmberJS-based app. It's totally async in nature, so testing it with Capybara is pain and misery. On the other hand, Ember provides a fantastic test suite out of the box which makes acceptance testing fun and effective.
Normally, either fixtures or backend mocks are used to acceptance-test Ember apps. But testing against mocks does not satisfy me at all:
It will not reveal possible API inconsistencies between the backend and the frontend, race conditions, etc.
It is impossible to test backend business logic this way. Such tests are not integration tests.
Finally, acceptance tests require persistency, so you have to replicate backend behavior in a mock. It's very tedious and you effectively end up implementing two backends.
So I want to test against the real backend! It's trivial to set up Ember to use a local backend instance for testing. But the problem is that the backend will persist its state between individual tests and even test sessions.
That's why I'm thinking of implementing a special public API in Rails:
The API is only available when Rails is run with a specific flag or an env var.
Rails runs in a non-testing mode, serving normal API calls as it would in production.
Before each test, the frontend calls the special API, telling Rails which database setup is needed for this specific test.
When a call to the special API is received, Rails cleans the database and populates it with the requested data. For example, to test item deletion from the cart, the database should have three items in the cart.
Rails finishes the API request, and the frontend starts the test.
Frontend runs test steps, using the normal backend API as it would in production: log in, create posts, comment on them. It will also try doing some forbidden things, e. g. edit posts while not logged in, exceed text length constraints, etc and verify whether the backend rejects forbidden actions.
When the frontend runs next test, it will call the special API again. Rails will discard the state produced by the previous test and set up a new one, for this specific test.
I'm a frontend dev with a sketchy knowledge of Rails. Factory Girl and Database Cleaner seem to be the right tools for the job, but there is absolutely no information how to use them outside Rails' normal test environment. I guess I need a controller or a Rails engine or something.
So the question is: how do I make an API in Rails that can be used by the frontend to tell Rails to set up a certain database state with a fixture factory, while Rails are running in a non-test mode i. e. serving REST API and not running through RSpec/Capybara/Cucumber?
Bonus feature: fixture factory properties should be defined on the frontend so that test code is stored in one place. Thus, the backend should be able to accept fixture factory properties via the special API. Defaults can still be defined in the backend's codebase.
I believe this could become an acceptance/integration testing best practice. If I manage to implement it, I promise to publish a gem.
May be something like this
config/routes.rb
namespace 'test_api' do
resource 'db_transaction', only: [:create, :destroy]
end if Rails.env.test?
controllers/test_api/db_transactions_controller.rb
require 'database_cleaner'
def create
DatabaseCleaner.start
end
def destroy
DatabaseCleaner.clean
end
I call a third party web service right now as part of my application. I am using the RestClient gem in order to do this. There are a ton of tools available to do the same thing, so that should not matter.
What I'm curious about is having good enough tests, nothing too fancy, where I can simulate how my application responds when the third party web service is unavailable for whatever reason. Be it that I exceeded a rate limit or a timeout due to network latency/complications, I just want to be able to take something like an HTTP status code and test what my application does in that event.
What's the best way to do this with Test::Unit? Right now the call to the third party service is encapsulated inside of one of my controllers. I have a simple module with some wrapper methods for the different end points of the remote service. I just want to make sure that my application does the right things when the service is or isn't available.
Is using an additional framework next to Test::Unit that can 'stub' the right way to go about doing this? Obviously I can't force a network timeout and starting to hack with things like IPtables just for tests is not worth the time. I'm sure this problem has been solved a million times as integrating things such as Facebook and Twitter into web applications is so popular these days. How do you test for failure when reaching those APIs in a robust/controlled format?
I would recommend using something like webmock to mock all of your http requests (not just to mock a failed request); it'll greatly speed up your test suite instead of having to actually hit the third party service every time you run the tests.
Webmock supports Rest Client and Test::Unit. Just put this code in your test/test_helper.rb file:
require 'webmock/test_unit'
As an example, to test a network timeout:
stub_request(:any, 'www.example.net').to_timeout
RestClient.post('www.example.net', 'abc') # ===> RestClient::RequestTimeout
railscast: 291 (subscriber only) talks about testing with VCR and rspec (i know, not it's not Test:Unit)
anyway you could look into using VCR for this sort of thing
I am using Steak and Capybara to do my integration tests. I also would like to test the serialization output (webservices using XML and JSON format) of my models. The problem is that JSON is only available by using a XML HTTP Request. So Capybara's visit method does not work. It also seems that I don't have access to xhr method in my acceptance tests (not sure why, because I require the normal spec_helper.rb in my acceptance_helper.rb).
How do you test web services? Using the xhr method (after requiring it somehow)? A special method with Capybara? Something else?
I also have some custom serialization (beside the normal my_object.to_xml). I guess it is better to do that in the model tests. Would you then still test the web service output of those custom serializations?
You can always create a "proxy" controller that makes requests to the web service and prints the result. It should be available only in test environment, obviously. Then your Capybara test visits the proxy and tests page content.
I'll keep it short, I've got a rails app which communicate with other apps, some using SOAP (non-rails apps of course...) and others with REST. I'm making integration tests to ensure that my endpoint wrapper classes have correct mappings and setup. However, they are executed by default by rake test which makes it slow and fragile. I wish to run unit tests frequently and integration tests "on-demand" only. How do you do that?
What're your preferences wrt such integration testing?
How deep do you unit test and/or mock?
Do you replicate whole SOAP or REST xml responses in stubs?
Do you create "external endpoint" integration tests at all?
Update Q: How to exclude a test-dir while running rake test ?
If you go by what the Rspec/Cucumber folks suggest, then the integration test level is an inappropriate place to mock your data, because in some respects, it defeats the purpose of the integration/acceptance test. However, you have to mock stuff like paypal transactions, right? In my current project, I am facing a lot of this, and here are some of the solutions I am implementing:
Tagging tests that wont work in certain contexts. In my example, lots of servers live behind firewalls and so my tests dont pass if I am at home and not using vpn. So, in cucumber I can tag these as #firewall and tell it to run tests that are not tagged firewall. I'm pretty sure Rspec 2.0 supports this feature as well.
Mocking service requests. Yah, its probably a bad idea, but I am at a loss on how to do it otherwise with any kind of predictability. I have a separate test suite to affirm that the services are running, and from my rails app, i am assuming they are working properly. An example of this would be LDAP. And yes, in these circumstances, I tend to use a real response and do something like. response = double('response') ; response.expects(:data).and_returns('my xml here')
I do think regardless of the complexity of the system that end point tests are really important. I am really enjoying cucumber, it provides me 95% of what I need to do in functional tests, and so I end up writing fewer of these tests and more of the entire workflow tests.
The solution is VCR.
Excluding endpoint integration tests from rake test and being able to isolate and run them with rake test:endpoints was solved with only a few lines of code. I have to admit though, I spent a whole lot of hours swearing and cursing. There should be more documentation and explanations in the railties source. Ruby code like that tend not to be very self-explanatory, IMO.
Well, here it goes:
create your task: lib/tasks/slow_tests.rake
require 'rails/test_unit/railtie'
desc "Runs all endpoint integration tests."
namespace :test do
#hooks on to the test task through the 'test:prepare'
#For details, check the railties gem (v3.0+) lib/rails/test_unit/testing.rake,
#look for "task :test" and "namespace :test"
TestTaskWithoutDescription.new(:endpoints => 'test:prepare') do |t|
t.libs << 'test'
t.pattern = 'test/endpoints/**/*_test.rb'
end
end
Now I may put my fragile endpoint integration tests in the test/enpoints directory, running them whenever I want (not often)
Note: this supposes test/unit or shoulda.
You should write a thin layer around external APIs (Facade/Wrapper) and use the vcr-gem to "stub" network calls.
You can get more information from my article on rails test architecture.