I'm using faraday and i have implemented a retry mechanism which is working as expected but i don't understand how can i test it on rspec.
Below is my retry mechanism:
def perform(url:)
max_retries = 3
retry_count = 0
delay = 1
begin
req = config_faraday
response = req.get(url)
JSON.parse(response.body)
rescue Faraday::Error => err
puts "Request failed. Retries left: #{max_retries - retry_count}"
sleep delay += retry_count
retry_count += 1
retry if retry_count < max_retries
ensure
if retry_count == max_retries
raise ApiError.new("Number of retries has been exhausted")
end
end
end
I'm not sure how to handle the case of when api is not working
I have written it by stubbing the get request and making the retry_count to max and then trying to hit the request:
context 'Api is not working' do
let(:retry_count){
3
}
before do
request = double(Faraday)
allow(request).to receive(:get).and_raise(Faraday::Error)
end
it "raise error after retries" do
expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
end
end
But i'm getting an error:
Products::Fetcher Api is not working raise error after retries
Failure/Error: expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
expected Products::ApiError but was not given a block
# ./spec/services/products/fetcher_spec.rb:34:in `block (3 levels) in <top (required)>'
Weird: your tests as such do not test any of the code of the perform. Which is sometimes good, but in this case they do not seem to test anything other than your stub.
For API testing I would suggest looking at a gem like vcr. But this would not work to reproduce the failing, then retrying and then succeeding.
To test that, from the code you showed, I would stub config_faraday to return a mock object, that would raise an error two times when get is called and return a body on the third time, for instance.
Wrong syntax for raise_error matcher
Correct is:
expect { raise StandardError }.to raise_error
Your example:
expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
So correct form should be:
expect{ Products::Fetcher.perform(url: url) }.to raise_error(Products::ApiError)
Related
I'm having RSpec which expect some exception to be raised. However, when invoking the method, I got error: expected Exception but nothing was raised
my code:
require 'rails_helper'
RSpec.describe ReportMailer, type: :mailer do
let(:csv_file) { "data\n1\n2" }
let(:filename) { 'dummy.csv' }
let(:options) do
{
filename: filename,
to: user.email,
subject: "Transaction Report - #{filename}"
}
end
it 'raise error' do
expect do
ReportMailer.notify(options, nil)
end.to raise_error(SomeExceptions::InvalidFile)
end
end
The problem is that if I just use normal expect call, let's say
expect(described_class.notify(dummy_options, nil)).to eq 1
RSpec show failure/error that I expect before:
Failures:
1) ReportMailer raise error
Failure/Error: raise SomeExceptions::InvalidFile
SomeExceptions::InvalidFile:
The file is invalid
# ./app/mailers/report_mailer.rb:5:in `notify'
# ./spec/mailers/report_mailer_spec.rb:37:in `block (2 levels) in <top (required)>'
My notify method is below:
require 'some_cms_exceptions'
class ReportMailer < ApplicationMailer
def notify(options, csv)
binding.pry
raise SomeExceptions::InvalidFile
validate(options)
attachments[options[:filename]] = { mime_type: 'text/csv', content: csv }
mail(to: options[:to], subject: options[:subject])
end
private
def validate(options)
raise SomeExceptions::InvalidMailOptions unless !options[:to].blank? && !options[:filename].blank?
end
end
I then put binding.pry in notify method and found out that:
if we use expect block, i.e. expect.{...}.to, notify method is not executed.
But if we use normal expect, i.e. expect(...).to ,notify method is executed.
May I know why it behaves like this ? Because other SO questions showed it works by using expect block.
Thank you
At line 5 you are raising SomeExceptions::InvalidFile exception while you are expecting different error in the expectation block
raise_error(SomeExceptions::InvalidMailOptions)
either you replace the exception you are expected or catch all exception using just raise_error without passing any error class(not recommended but for sake of testing).
The answer is as commented by #amit-patel, we need to add deliver_now to really execute mailer RSpec test case.
it 'should send email ' do
expect { ReportMailer.notify(options, nil).deliver_now }.to raise_error(SomeExceptions::InvalidMailOptions)
end
I am trying to test sending an email with Sidekiq and I am having an error which means it doesn't enqueue since my test shows it didn't change from 0 to 1 in size. Could it be I am missing something out or how can I fix this?
Error
expected `Array#size` to have changed by 1, but was changed by 0
0) Sending Mail sends email to sidekiq
Failure/Error:
expect do
OrderPointsMailer.paid_order_email(customer_detail.id).deliver_later
end.to change(Sidekiq::Worker.jobs, :size).by(1)
expected `Array#size` to have changed by 1, but was changed by 0
# ./spec/requests/order_points_mailer_spec.rb:9:in `block (2 levels) in <top (required)>'
1 example, 1 failure, 0 passed
binding.pry
This shows Sidekiq::Worker.jobs is empty array and it is understandable why it failed but I do not know how I could fix this.
[1] pry(#<RSpec::ExampleGroups::SendingMailWithSidekiq>)>Sidekiq::Worker.jobs
=> []
app/jobs/order_points_job.rb
# frozen_string_literal: true
class OrderPointsJob < ApplicationJob
def perform(customer_email)
customer_detail = CustomerDetail.find_by(email: customer_email)
return unless customer_detail
OrderPointsMailer.paid_order_email(customer_detail.id).deliver_later # This call send email to sidekiq process
end
end
RSpec
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe 'Sending Mail with Sidekiq', type: :request do
it 'sends email to sidekiq' do
customer_detail = create(:customer_detail)
expect do
OrderPointsMailer.paid_order_email(customer_detail.id).deliver_later
end.to change(Sidekiq::Worker.jobs, :size).by(1)
end
end
rails_helper.rb
require 'sidekiq/testing'
Sidekiq::Testing.fake!
You probably should use the method count instead of size:
.to change(Sidekiq::Worker.jobs.reload, :count).by(1)
Take a look at the size implementation:
From: /home/toptal/.rvm/gems/ruby-2.5.5/gems/activerecord-6.0.0/lib/active_record/relation.rb # line 260:
Owner: ActiveRecord::Relation
Visibility: public
Number of lines: 3
def size
loaded? ? #records.length : count(:all)
end
Also take a look on the matcher implementation (https://github.com/rspec/rspec-expectations/blob/master/lib/rspec/matchers/built_in/change.rb#L82:L85)
I'm guessing that the receiver (Sidekiq::Worker.jobs in your case) will be evaluated once, and then the size would be called twice on the same objects collection, without reloading.
One other approach you could take would be
.to change { Sidekiq::Worker.jobs.reload.size }.by(1)
But this would be a bit wasteful in case you have more records in the DB, because each time the AR objects will need to be created (and they're not used)
I have been attempting to test this method for the past few days with no luck.
Another thing I'd like to be able to do is rescue the error that bubbles up after the final retry attempt is made.
Please see my comments and code snippets below.
Source code for retry_on is here as well for context.
Here's the sample code and tests:
my_job.rb
retry_on Exception, wait: 2.hours, attempts: 3 do |job, exception|
# some kind of rescue here after job.exceptions == 3
# then notify Bugsnag of failed final attempt.
end
def perform(an_object)
an_object.does_something
end
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
expect(AnObject).to receive(:does_something).and_raise { Exception }.exactly(3).times
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts: 3).exactly(3).times
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
The test failure response:
1) MyJob.perform receives retry_on 3 times
Failure/Error: expect(job).to receive(:retry_on).with(wait: 4.hours, attempts: 3).exactly(3).times
(MyJob (class)).retry_on({:wait=>2 hours, :attempts=>3})
expected: 3 times with arguments: ({:wait=>2 hours, :attempts=>3})
received: 0 times
# ./spec/jobs/my_job_spec.rb:38:in `block (4 levels) in <top (required)>'
# ./spec/rails_helper.rb:48:in `block (3 levels) in <top (required)>'
# ./spec/rails_helper.rb:47:in `block (2 levels) in <top (required)>'
I've also tried making the job a double and stubbing the retry_on method and that doesn't work either.
I've also tried using Timecop to fast forward the wait time and tests are still failing:
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
expect(AnObject).to receive(:does_something).and_raise { Exception }.exactly(3).times
Timecop.freeze(Time.now + 8.hours) do
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts: 3).exactly(3).times
end
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
It IS a class method of ActiveJob and I've confirmed this in a byebug terminal that this is the case with my job class.
Shouldn't this test work? It's expecting the class to receive the class method with certain arguments. My byebug gets hit when I put it in the retry_on block as well so I know that the method is getting called multiple times.
It's almost as if it's being called on a different class which is very confusing and I don't think is the case but I'm at the end of my rope with this one.
I almost resolved the issue by decoupling my tests from testing the retry_on rails logic itself to testing my business logic around it. This way is better as well in the case that rails ever changes the retry_on logic.
HOWEVER, this does NOT work for more than one test case. If you use this with more than one case, the last test will break and say it has performed more jobs than expected.
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
allow(AnObject).to receive(:does_something).and_raise { Exception }
expect(AnObject).to receive(:does_something).exactly(3).times
expect(Bugsnag).to receive(:notify).with(Exception).once
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
my_job.rb
retry_on Exception, wait: , attempts: 3 do |job, exception|
Bugsnag.notify(exception)
end
def perform(an_object)
an_object.does_something
end
Any help/insight on this would be greatly appreciated.
Would also love a recommendation on how to handle the bubbled up exception after max attempts too. I'm thinking of raising an error within the retry_on block and then have discard_on trigger for the error that's raised.
Thank you wonderful Stack Overflow community!
This is the format of specs needed for retry_on that finally worked for me:
it 'receives retry_on 10 times' do
allow_any_instance_of(MyJob).to receive(:perform).and_raise(MyError.new(nil))
allow_any_instance_of(MyJob).to receive(:executions).and_return(10)
expect(Bugsnag).to receive(:notify)
MyJob.perform_now(an_object)
end
it 'handles error' do
allow_any_instance_of(MyJob).to receive(:perform).and_raise(MyError.new(nil))
expect_any_instance_of(MyJob).to receive(:retry_job)
perform_enqueued_jobs do
MyJob.perform_later(an_object)
end
end
For the first case,
executions is an ActiveJob method that gets run, set and checked every time retry_on is executed. We mock it to return 10 and then expect it to call Bugsnag. retry_on only calls what you gave it in the block once all the attempts have been met. So this works.
For the second case,
Then mock the error to raise for the job instance.
Next we check that it's correctly receiving retry_job (which retry_on calls under the hood) to confirm it's doing the right thing.
Then we wrap the perform_later call in the minitest perform_enqueued_jobs block and call it a day.
The following works fine for me, also for multiple testcases and for testing side effects of the retry_on block.
RSpec.describe MyJob, type: :job do
include ActiveJob::TestHelper
context 'when `MyError` is raised' do
before do
allow_any_instance_of(described_class).to receive(:perform).and_raise(MyError.new)
end
it 'makes 4 attempts' do
assert_performed_jobs 4 do
described_class.perform_later rescue nil
end
end
it 'does something in the `retry_on` block' do
expect(Something).to receive(:something)
perform_enqueued_jobs do
described_class.perform_later rescue nil
end
end
end
end
Note that rescue nil (or some form of rescue) is required if you let exceptions bubble up at the end.
Note that perform_now doesn't count as "enqueued job". So doing described_class.perform_now results in one less attempts counted by assert_performed_jobs.
IMHO you should leave the testing of ActiveJob with the rails team.
You only need to make sure you're configuring the job properly:
it 'retries the job 10 times with 2 minutes intervals' do
allow(MyJob).to receive(:retry_on)
load 'app/path/to/job/my_job.rb'
expect(MyJob).to have_received(:retry_on)
.with(
Exception,
wait: 2.minutes,
attempts: 10
)
end
In the first spec
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts:3).exactly(3).times
This will never gonna work since class method retry_on will be called on class initialization phase, ie when loading that class into memory, not while executing a spec
In the second spec, you tried to make it work using timecop but still failed for the same reason
Third spec is relatively more realistic, but
assert_performed_jobs 3
won't work without passing the block
Something like
assert_performed_jobs 2 do
//call jobs from here
end
While I agree with #fabriciofreitag that one shouldn't need to test the internals of an external library, I think there's definitely value in confirming that your retry_on blocks are configured properly. This setup worked for me without having to worry about how ActiveJob manages the retries.
# app/jobs/my_job.rb
retry_on TimeoutError,
wait: :exponentially_longer,
attempts: 3
# spec/jobs/my_job_spec.rb
describe "error handling" do
before do
ActiveJob::Base.queue_adapter.perform_enqueued_at_jobs = true
end
context "when TimeoutError is raised" do
it "retries timed-out requests" do
expect(client).to receive(:connect).ordered.and_raise(TimeoutError)
expect(client).to receive(:connect).ordered.and_call_original
described_class.perform_now
end
it "retries three times before re-raising" do
expect(client).to receive(:connect)
.exactly(3).times.and_raise(TimeoutError)
expect { described_class.perform_now }.to raise_error(TimeoutError)
end
end
end
When debugging failing integration tests, I keep running into the same problem where the exceptions raised in my code are suppressed and not shown in the testing output.
For example, for the following controller and test:
class RegistrationController::ApplicationController
def create
# some code that raises an exception
end
end
class RegistrationFlowTest < ActionDispatch::IntegrationTest
test 'user registers successfully' do
post sign_up_path, params: { username: 'arnold', password: '123' }
assert_response :success
end
end
The output is something like
Minitest::Assertion: Expected response to be a <2XX: success>, but was a <500: Internal Server Error>
Is there a way to see the exact raised exception? Instead of just the difference of HTTP response code?
Thanks!
Simon
My recommended approach to this issue would be to actually parse the response provided by Rails (at least by default in test and development environments) which includes the stacktrace for the error and handle that in the case that your test fails. This has the advantage that it won't output the stacktrace when errors are raised that don't result in failing tests (e.g. scenarios where you are intentionally testing how failures are handled).
This little module that I've crafted will allow you to call assert_response_with_errors to assert the response to a call but output the exception message and stack trace in a readable format when the response is not what you expected.
module ActionDispatch
module Assertions
module CustomResponseAssertions
# Use this method when you want to assert a response body but also print the exception
# and full stack trace to the test console.
# Useful when you are getting errors in integration tests but don't know what they are.
#
# Example:
# user_session.post create_gene_path, params: {...}
# user_session.assert_response_with_errors :created
def assert_response_with_errors(type, message = nil)
assert_response(type, message)
rescue Minitest::Assertion => e
message = e.message
message += "\nException message: #{#response.parsed_body[:exception]}"
stack_trace = #response.parsed_body[:traces][:'Application Trace'].map { |line| line[:trace] }.join "\n"
message += "\nException stack trace start"
message += "\n#{stack_trace}"
message += "\nException stack trace end"
raise Minitest::Assertion, message
end
end
end
end
To use this, you need to include it into ActionDispatch::Assertions before Rails has loaded its stack in your test_helper.rb. So just prepend the include into your test_helper.rb, like this:
ActionDispatch::Assertions.include ActionDispatch::Assertions::CustomResponseAssertions
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
...
This will happen because Rails controllers by default handle exceptions and raise the 500 status, making the exceptions invisible to the test suite (which is very helpful if errors are raised in unit tests of a model). Options for disabling this in your test suite, or alternative workarounds, are discussed here.
The key lines of code from that link, which should be added to test/integration/integration_test_helper.rb:
ActionController::Base.class_eval do
def perform_action
perform_action_without_rescue
end
end
Dispatcher.class_eval do
def self.failsafe_response(output, status, exception = nil)
raise exception
end
end
EDIT: I've noticed that that link is quite old now. I'm really familiar with Rack, so whilst the first block looks ok to me, I'm not sure if the second will still be current. You might need to have a look at the relevant current Rails guide if it needs bringing up to date.
I use the capybara-webkit gem to scrape data from certain pages in my Rails application. I've noticed, what seems to be "random" / "sporadic", that the application will crash with the following error:
Capybara::Webkit::ConnectionError: /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/bin/webkit_server failed to start.
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/server.rb:56:in `parse_port'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/server.rb:42:in `discover_port'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/server.rb:26:in `start'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/connection.rb:67:in `start_server'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/connection.rb:17:in `initialize'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/driver.rb:16:in `new'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/driver.rb:16:in `initialize'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit.rb:15:in `new'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit.rb:15:in `block in <top (required)>'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-2.7.1/lib/capybara/session.rb:85:in `driver'
from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-2.7.1/lib/capybara/session.rb:233:in `visit'
It happens even after it's already connected and accessed a website multiple times before. Here's a code snippet of what I'm using currently...
if site.url.present?
begin
# Visit the URL
session = Capybara::Session.new(:webkit)
session.visit(site.url) # here is where the error occurs...
document = Nokogiri::HTML.parse(session.body)
# Load configuration options for Development Group
roster_table_selector = site.development_group.table_selector
header_row_selector = site.development_group.table_header_selector
row_selector = site.development_group.table_row_selector
row_offset = site.development_group.table_row_selector_offset
header_format_type = site.config_header_format_type
# Get the Table and Header Row for processing
roster_table = document.css(roster_table_selector)
header_row = roster_table.css(header_row_selector)
header_hash = retrieve_headers(header_row, header_format_type)
my_object = process_rows(roster_table, header_hash, site, row_selector, row_offset)
rescue ::Capybara::Webkit::ConnectionError => e
raise e
rescue OpenURI::HTTPError => e
if e.message == '404 Not Found'
raise "404 Page not found..."
else
raise e
end
end
end
I've even thought perhaps I don't find out why it's happening necessarily - but just recover when it does. So I was going to do a "retry" in the rescue block for the error but it appears the server is just down - so I get the same result when retrying. Perhaps someone knows of a way I can check if the server is down and restart it then perform a retry? Thanks for the help!
So after further investigating it appears that I was generating a new Capybara::Session for each iteration of my loop. I moved it outside of the loop and also added Capybara.reset_sessions! at the end of my loop. Not sure if that helps with anything -- but the issue seems to have been resolved. I'll monitor it for the next hour or so. Below is an example of my ActiveJob code now...
class ScrapeJob < ActiveJob::Base
queue_as :default
include Capybara::DSL
def perform(*args)
session = Capybara::Session.new(:webkit)
Site.where(config_enabled: 1).order(:code).each do |site|
process_roster(site, session)
Capybara.reset_sessions!
end
end
def process_roster(site, session)
if site.roster_url.present?
begin
# Visit the Roster URL
session.visit(site.roster_url)
document = Nokogiri::HTML.parse(session.body)
# processing code...
# pass the session that was created as the final parameter..
my_object = process_rows( ..., session)
rescue ::Capybara::Webkit::ConnectionError => e
raise e
rescue OpenURI::HTTPError => e
if e.message == '404 Not Found'
raise "404 Page not found..."
else
raise e
end
end
end
end
end