With my Rails project, I would like to write tests non-ideal conditions such as lack of internet connection or timeouts. For example, I am using a gem to contact an API and would like to make sure that I handle the error correctly if there is a connection issue between my app and the external API.
I can do this already by making a fixture with VCR and removing the response from the "cassette". However, this has drawbacks:
It has to be done manually.
The cassettes can not be gitignored if I am working with a team(which I am).
How can I simply create a block in my RSpec tests that will prevent external connections, simulating the lack of an internet connection?
Per this thought-bot article
https://robots.thoughtbot.com/how-to-stub-external-services-in-tests#disable-all-remote-connections
# spec/spec_helper.rb
require 'webmock/rspec'
WebMock.disable_net_connect!(allow_localhost: true)
I've never tried to do this, but perhaps you could use webmock to stub out all requests.
before do
stub_request(:any, /.*/).to_return(body: "errors", status: 422)
end
More info on stubbing external services.
Related
I have a Rails application that has changed from user authentication via devise to accessing a remote API for authentication. For the test suite, this requires that I mock the https request for authentication and return a valid response. We are using Cucumber and Capybara for testing and I am attempting to use the webmock Gem to mock the login response.
The login is initiated by a button click on a form (Capybara action 'click_button').
Unfortunately, although webmock appears to be installed correctly (I am able to make a Net::HTTP POST request and this is recognized by Webmock), the POST to the remote authorization facility is not being captured by webmock. I know that the form POST is making its way to the controller, because the stacktrace shows that the POST is being executed in the controller as it should and the error message is "Errno::ECONNREFUSED at ... Failed to open TCP connection".
I have tried:
WebMock.stub_request(:any, '127.0.0.1').to_return(body: {STATUSCODE: 1}.to_json)
and
WebMock::stub_request(:any, '127.0.0.1').to_return(body: {STATUSCODE: 1}.to_json)
and
stub_request(:any, '127.0.0.1').to_return(body: {STATUSCODE: 1}.to_json)
I have tried putting the stub_request call in the devise_steps.rb file, just before the "click_button" command, as well as in the features/support/env.rb and features/support/webmock.rb file.
I assume that what I am doing is so common that it has to be possible, but I have found nothing that indicates why the stub_request is not successful.
So the domain of the remote API for authentication is localhost? So it would be running on the same server with a different port? Then you have to mock the address with the port.
For instance your Rails app is running on port 80 and your auth API is running on 8080 then you have to do this.
stub_request(:any, '127.0.0.1:8080').to_return(body: {STATUSCODE: 1}.to_json)
I think Webmock should support different ports but I'm not 100% sure. Also have you set WebMock.disable_net_connect!(allow_localhost: true)?
https://github.com/bblimke/webmock#external-requests-can-be-disabled-while-allowing-localhost
Edit:
If this is not working, I suggest you make the API URL configurable and e.g. set it in tests to auth-api.com and then mock this.
The root cause of the problem was that the path on the stub_request was not complete. I had misread the webmock documentation....
When I disable Webmock or VCR I get the following error in my Rspec tests. Apparently it's looking for an /__identify__ route.
WebMock::NetConnectNotAllowedError:
Real HTTP connections are disabled. Unregistered request: GET http://127.0.0.1:51768/__identify__ with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'User-Agent'=>'Ruby'}
A bit of googling shows that this is something specific to Capybara, but I wasn't able to find out what exactly it's trying to do and why it might need to be stubbed.
Thanks!
You shouldn't be stubbing it - you should be allowing it (like all requests to your app). Its a route added via middleware Capybara installs when it starts up the "server" thread running your app. Capybara needs it to know the app has started up and is ready to go.
For VCR you can configure this way:
VCR.configure do |config|
// more VCR configs here
config.ignore_request do |request|
URI(request.uri).host == "127.0.0.1"
end
end
My VCR config is:
VCR.configure do |c|
c.configure_rspec_metadata!
c.cassette_library_dir = 'spec/cassettes'
c.hook_into :webmock
c.ignore_localhost = true
end
And a test example is:
it "creates a build", :vcr => {:cassette_name => "build/feature/create"} do
visit new_build_path(build)
fill_in("build_name", :with => "Test Build")
click_button("Create Build")
build = Build.first
page.should have_content("Build was successfully created.")
current_path.should == build_path(hub)
end
When running this test it calls out to several 3rd Party API's which requests get recorded via VCR. The problem I am having is that it appears that VCR is using requests from other cassettes while running which is causing intermittent failures for certain tests. I have checked the cassettes and sometimes (depending on order is how it appears) all requests will be recorded and played back perfectly. It is worth noting this is when the whole suite is run, they always work when run by themselves. I do not share cassettes between the specs that fail, the only thing that is shared is some common requests to the API's and I force the naming of the cassettes just to make sure it is using the correct ones. I hope this makes sense...
My main question is what can be causing this problem? When using record => :new_episodes the tests work perfectly as well but not when using the record => :once mode. This may be okay given the situation but I want to make sure I am not creating unnecessary requests and from my understanding record => :once should work given the requests for each spec should be isolated.
I know this may be hard to answer without more information so let me know if any will help. Thanks in advance!
As you said, it's hard to answer from the little bit of information you've given. I can give some suggestions for helping to troubleshoot, though.
You mention that you're using a capybara JS driver. Capybara's JS drivers are asynchronous, which means that you test thread may continue on before your app has processed a particular request -- so if the test script continues on and ejects or inserts a new VCR cassette, and then your app makes an HTTP request -- it could lead to race conditions.
VCR has a debug_logger option that will give you lots of insight into exactly what VCR is doing. It may answer your question.
Good luck!
I would like to run RSpec to test my code both when I'm connected to the web and when I'm not.
Unfortunately there are some tests in the application which are dependent on having a live connection - email sending, Facebook integration etc.
Is there a best-practice way to create an online/offline testing environment or is this bad practice? Judging by how little I can find about this on the web I'm guessing the latter.
Normally in situations like that you would mock the parts of the code that connect outside your application. This allows you to test against the expected results from the service/system you are connecting to. It's also quicker to run tests.
There's a brief tutorial on mocking with rspec here but I'm sure you can find plenty yourself.
For testing that emails get sent there are other approaches if you are sending through ActionMailer. There's a section on that in the rails testing guide.
EDIT (in response to comment):
You could put a method in TestHelper to only run tests when you are online. Something like:
def when_online
if test_remote_connectivity
yield
else
puts "Skipping test offline."
end
end
Then you can call it like:
def test_facebook
when_online do
.....
end
end
Not sure I entirely advocate it but it might do what you want!
You could use webmock inside the tests/specs you don't want connecting to the remote resource.
I'll keep it short, I've got a rails app which communicate with other apps, some using SOAP (non-rails apps of course...) and others with REST. I'm making integration tests to ensure that my endpoint wrapper classes have correct mappings and setup. However, they are executed by default by rake test which makes it slow and fragile. I wish to run unit tests frequently and integration tests "on-demand" only. How do you do that?
What're your preferences wrt such integration testing?
How deep do you unit test and/or mock?
Do you replicate whole SOAP or REST xml responses in stubs?
Do you create "external endpoint" integration tests at all?
Update Q: How to exclude a test-dir while running rake test ?
If you go by what the Rspec/Cucumber folks suggest, then the integration test level is an inappropriate place to mock your data, because in some respects, it defeats the purpose of the integration/acceptance test. However, you have to mock stuff like paypal transactions, right? In my current project, I am facing a lot of this, and here are some of the solutions I am implementing:
Tagging tests that wont work in certain contexts. In my example, lots of servers live behind firewalls and so my tests dont pass if I am at home and not using vpn. So, in cucumber I can tag these as #firewall and tell it to run tests that are not tagged firewall. I'm pretty sure Rspec 2.0 supports this feature as well.
Mocking service requests. Yah, its probably a bad idea, but I am at a loss on how to do it otherwise with any kind of predictability. I have a separate test suite to affirm that the services are running, and from my rails app, i am assuming they are working properly. An example of this would be LDAP. And yes, in these circumstances, I tend to use a real response and do something like. response = double('response') ; response.expects(:data).and_returns('my xml here')
I do think regardless of the complexity of the system that end point tests are really important. I am really enjoying cucumber, it provides me 95% of what I need to do in functional tests, and so I end up writing fewer of these tests and more of the entire workflow tests.
The solution is VCR.
Excluding endpoint integration tests from rake test and being able to isolate and run them with rake test:endpoints was solved with only a few lines of code. I have to admit though, I spent a whole lot of hours swearing and cursing. There should be more documentation and explanations in the railties source. Ruby code like that tend not to be very self-explanatory, IMO.
Well, here it goes:
create your task: lib/tasks/slow_tests.rake
require 'rails/test_unit/railtie'
desc "Runs all endpoint integration tests."
namespace :test do
#hooks on to the test task through the 'test:prepare'
#For details, check the railties gem (v3.0+) lib/rails/test_unit/testing.rake,
#look for "task :test" and "namespace :test"
TestTaskWithoutDescription.new(:endpoints => 'test:prepare') do |t|
t.libs << 'test'
t.pattern = 'test/endpoints/**/*_test.rb'
end
end
Now I may put my fragile endpoint integration tests in the test/enpoints directory, running them whenever I want (not often)
Note: this supposes test/unit or shoulda.
You should write a thin layer around external APIs (Facade/Wrapper) and use the vcr-gem to "stub" network calls.
You can get more information from my article on rails test architecture.