My VCR config is:
VCR.configure do |c|
c.configure_rspec_metadata!
c.cassette_library_dir = 'spec/cassettes'
c.hook_into :webmock
c.ignore_localhost = true
end
And a test example is:
it "creates a build", :vcr => {:cassette_name => "build/feature/create"} do
visit new_build_path(build)
fill_in("build_name", :with => "Test Build")
click_button("Create Build")
build = Build.first
page.should have_content("Build was successfully created.")
current_path.should == build_path(hub)
end
When running this test it calls out to several 3rd Party API's which requests get recorded via VCR. The problem I am having is that it appears that VCR is using requests from other cassettes while running which is causing intermittent failures for certain tests. I have checked the cassettes and sometimes (depending on order is how it appears) all requests will be recorded and played back perfectly. It is worth noting this is when the whole suite is run, they always work when run by themselves. I do not share cassettes between the specs that fail, the only thing that is shared is some common requests to the API's and I force the naming of the cassettes just to make sure it is using the correct ones. I hope this makes sense...
My main question is what can be causing this problem? When using record => :new_episodes the tests work perfectly as well but not when using the record => :once mode. This may be okay given the situation but I want to make sure I am not creating unnecessary requests and from my understanding record => :once should work given the requests for each spec should be isolated.
I know this may be hard to answer without more information so let me know if any will help. Thanks in advance!
As you said, it's hard to answer from the little bit of information you've given. I can give some suggestions for helping to troubleshoot, though.
You mention that you're using a capybara JS driver. Capybara's JS drivers are asynchronous, which means that you test thread may continue on before your app has processed a particular request -- so if the test script continues on and ejects or inserts a new VCR cassette, and then your app makes an HTTP request -- it could lead to race conditions.
VCR has a debug_logger option that will give you lots of insight into exactly what VCR is doing. It may answer your question.
Good luck!
Related
It's the first test im writing that requires :js => true so I installed selenium.
I am always able to login with some credentials I create before but unable to log in when I run them through selenium.
A sanity check to see if I'm still creating my user (recruiter)
puts "RECRUITER NAME => #{Recruiter.first.email}"
returns
RECRUITER NAME => company1#test.com
I always set my password to "testtesttest"
But it always fails to login.
Is it using a different (empty?) database? Am I missing some settings that are required?
Note => Logging in with the exact same file works flawless if I remove :js => true. But somehow it breaks on this step.
When performing js tests the app being tested runs in a different thread than the test, which means they no longer share the same database connection - see https://github.com/jnicklas/capybara#transactions-and-database-setup. There are ways to force the two threads to share the connection, but it ends up being flaky on some edge cases and limiting what database access you can do at specific times in your test, and really isn't something a beginner wants to deal with. The better/easiest solution is to disable transactional testing and use truncation or deletion to manage the database state. The easiest way to do that is to use database_cleaner and setup a config that will swap to the needed strategy for each test - https://github.com/DatabaseCleaner/database_cleaner#rspec-with-capybara-example
With my Rails project, I would like to write tests non-ideal conditions such as lack of internet connection or timeouts. For example, I am using a gem to contact an API and would like to make sure that I handle the error correctly if there is a connection issue between my app and the external API.
I can do this already by making a fixture with VCR and removing the response from the "cassette". However, this has drawbacks:
It has to be done manually.
The cassettes can not be gitignored if I am working with a team(which I am).
How can I simply create a block in my RSpec tests that will prevent external connections, simulating the lack of an internet connection?
Per this thought-bot article
https://robots.thoughtbot.com/how-to-stub-external-services-in-tests#disable-all-remote-connections
# spec/spec_helper.rb
require 'webmock/rspec'
WebMock.disable_net_connect!(allow_localhost: true)
I've never tried to do this, but perhaps you could use webmock to stub out all requests.
before do
stub_request(:any, /.*/).to_return(body: "errors", status: 422)
end
More info on stubbing external services.
My rails app talks to a couple different APIs, and I'd like to write tests to make sure that errors are being handled properly in case there is a problem with a request. For example, if the service is down and not reachable.
Is there a way I can simulate an error coming back from a service even though it might be fine during the test?
The webmock gem helps you with this. For your specific question scroll down to "Response with custom status message".
stub_request(:any, "www.example.com").to_return(:status => [500, "Internal Server Error"])
req = Net::HTTP::Get.new("/")
Net::HTTP.start("www.example.com") { |http| http.request(req) }.message # ===> "Internal Server Error"
I've used this gem with Rspec and Test::Unit. The biggest drawback is that the matching logic which decides what requests apply to a particular mock doesn't always work as you'd expect. It's a minor gripe though. The gem is a net-positive, and much better than trying to hit a live service from a test case.
You can use a mock instead of the real service, and make it give back an error.
I would like to run RSpec to test my code both when I'm connected to the web and when I'm not.
Unfortunately there are some tests in the application which are dependent on having a live connection - email sending, Facebook integration etc.
Is there a best-practice way to create an online/offline testing environment or is this bad practice? Judging by how little I can find about this on the web I'm guessing the latter.
Normally in situations like that you would mock the parts of the code that connect outside your application. This allows you to test against the expected results from the service/system you are connecting to. It's also quicker to run tests.
There's a brief tutorial on mocking with rspec here but I'm sure you can find plenty yourself.
For testing that emails get sent there are other approaches if you are sending through ActionMailer. There's a section on that in the rails testing guide.
EDIT (in response to comment):
You could put a method in TestHelper to only run tests when you are online. Something like:
def when_online
if test_remote_connectivity
yield
else
puts "Skipping test offline."
end
end
Then you can call it like:
def test_facebook
when_online do
.....
end
end
Not sure I entirely advocate it but it might do what you want!
You could use webmock inside the tests/specs you don't want connecting to the remote resource.
I'll keep it short, I've got a rails app which communicate with other apps, some using SOAP (non-rails apps of course...) and others with REST. I'm making integration tests to ensure that my endpoint wrapper classes have correct mappings and setup. However, they are executed by default by rake test which makes it slow and fragile. I wish to run unit tests frequently and integration tests "on-demand" only. How do you do that?
What're your preferences wrt such integration testing?
How deep do you unit test and/or mock?
Do you replicate whole SOAP or REST xml responses in stubs?
Do you create "external endpoint" integration tests at all?
Update Q: How to exclude a test-dir while running rake test ?
If you go by what the Rspec/Cucumber folks suggest, then the integration test level is an inappropriate place to mock your data, because in some respects, it defeats the purpose of the integration/acceptance test. However, you have to mock stuff like paypal transactions, right? In my current project, I am facing a lot of this, and here are some of the solutions I am implementing:
Tagging tests that wont work in certain contexts. In my example, lots of servers live behind firewalls and so my tests dont pass if I am at home and not using vpn. So, in cucumber I can tag these as #firewall and tell it to run tests that are not tagged firewall. I'm pretty sure Rspec 2.0 supports this feature as well.
Mocking service requests. Yah, its probably a bad idea, but I am at a loss on how to do it otherwise with any kind of predictability. I have a separate test suite to affirm that the services are running, and from my rails app, i am assuming they are working properly. An example of this would be LDAP. And yes, in these circumstances, I tend to use a real response and do something like. response = double('response') ; response.expects(:data).and_returns('my xml here')
I do think regardless of the complexity of the system that end point tests are really important. I am really enjoying cucumber, it provides me 95% of what I need to do in functional tests, and so I end up writing fewer of these tests and more of the entire workflow tests.
The solution is VCR.
Excluding endpoint integration tests from rake test and being able to isolate and run them with rake test:endpoints was solved with only a few lines of code. I have to admit though, I spent a whole lot of hours swearing and cursing. There should be more documentation and explanations in the railties source. Ruby code like that tend not to be very self-explanatory, IMO.
Well, here it goes:
create your task: lib/tasks/slow_tests.rake
require 'rails/test_unit/railtie'
desc "Runs all endpoint integration tests."
namespace :test do
#hooks on to the test task through the 'test:prepare'
#For details, check the railties gem (v3.0+) lib/rails/test_unit/testing.rake,
#look for "task :test" and "namespace :test"
TestTaskWithoutDescription.new(:endpoints => 'test:prepare') do |t|
t.libs << 'test'
t.pattern = 'test/endpoints/**/*_test.rb'
end
end
Now I may put my fragile endpoint integration tests in the test/enpoints directory, running them whenever I want (not often)
Note: this supposes test/unit or shoulda.
You should write a thin layer around external APIs (Facade/Wrapper) and use the vcr-gem to "stub" network calls.
You can get more information from my article on rails test architecture.