Context:
I have a job that catch some errors with these configuration
class CheckBankStatusJob < BaseJob
discard_on(ConnectionErrors::Error) do
Logger.new(STDOUT).warn("Connection Error!")
end
retry_on(
BankError::NoResponseData,
StandardError,
wait: 5.seconds,
)
end
ConnectionErrors inherit from StandardError
BankError::NoResponseData inherit from StandardError
i want to test the discard_on event in rspec
it "logs down when RemoteErrors::Error" do
allow_any_instance_of(described_class).to receive(:perform).and_raise(ConnectionErrors::Error)
expect_any_instance_of(Logger).to receive(:warn).with("Connection Error!")
described_class.perform_now
end
it never go to discard_on block, instead it goes to retry_on block.
errors
Failure/Error:
expected: 1 time with arguments: xxx
received: 0 times
Does retry_on get executed first? or it is because the ConnectionErrors get inherited from StandardError?
thanks in advance
I think you need to replace "perform" with "perform_now" to raise ConnectionErrors::Error
allow_any_instance_of(described_class).to receive(:perform_now).and_raise(ConnectionErrors::Error)
Related
I'm having RSpec which expect some exception to be raised. However, when invoking the method, I got error: expected Exception but nothing was raised
my code:
require 'rails_helper'
RSpec.describe ReportMailer, type: :mailer do
let(:csv_file) { "data\n1\n2" }
let(:filename) { 'dummy.csv' }
let(:options) do
{
filename: filename,
to: user.email,
subject: "Transaction Report - #{filename}"
}
end
it 'raise error' do
expect do
ReportMailer.notify(options, nil)
end.to raise_error(SomeExceptions::InvalidFile)
end
end
The problem is that if I just use normal expect call, let's say
expect(described_class.notify(dummy_options, nil)).to eq 1
RSpec show failure/error that I expect before:
Failures:
1) ReportMailer raise error
Failure/Error: raise SomeExceptions::InvalidFile
SomeExceptions::InvalidFile:
The file is invalid
# ./app/mailers/report_mailer.rb:5:in `notify'
# ./spec/mailers/report_mailer_spec.rb:37:in `block (2 levels) in <top (required)>'
My notify method is below:
require 'some_cms_exceptions'
class ReportMailer < ApplicationMailer
def notify(options, csv)
binding.pry
raise SomeExceptions::InvalidFile
validate(options)
attachments[options[:filename]] = { mime_type: 'text/csv', content: csv }
mail(to: options[:to], subject: options[:subject])
end
private
def validate(options)
raise SomeExceptions::InvalidMailOptions unless !options[:to].blank? && !options[:filename].blank?
end
end
I then put binding.pry in notify method and found out that:
if we use expect block, i.e. expect.{...}.to, notify method is not executed.
But if we use normal expect, i.e. expect(...).to ,notify method is executed.
May I know why it behaves like this ? Because other SO questions showed it works by using expect block.
Thank you
At line 5 you are raising SomeExceptions::InvalidFile exception while you are expecting different error in the expectation block
raise_error(SomeExceptions::InvalidMailOptions)
either you replace the exception you are expected or catch all exception using just raise_error without passing any error class(not recommended but for sake of testing).
The answer is as commented by #amit-patel, we need to add deliver_now to really execute mailer RSpec test case.
it 'should send email ' do
expect { ReportMailer.notify(options, nil).deliver_now }.to raise_error(SomeExceptions::InvalidMailOptions)
end
I have been attempting to test this method for the past few days with no luck.
Another thing I'd like to be able to do is rescue the error that bubbles up after the final retry attempt is made.
Please see my comments and code snippets below.
Source code for retry_on is here as well for context.
Here's the sample code and tests:
my_job.rb
retry_on Exception, wait: 2.hours, attempts: 3 do |job, exception|
# some kind of rescue here after job.exceptions == 3
# then notify Bugsnag of failed final attempt.
end
def perform(an_object)
an_object.does_something
end
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
expect(AnObject).to receive(:does_something).and_raise { Exception }.exactly(3).times
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts: 3).exactly(3).times
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
The test failure response:
1) MyJob.perform receives retry_on 3 times
Failure/Error: expect(job).to receive(:retry_on).with(wait: 4.hours, attempts: 3).exactly(3).times
(MyJob (class)).retry_on({:wait=>2 hours, :attempts=>3})
expected: 3 times with arguments: ({:wait=>2 hours, :attempts=>3})
received: 0 times
# ./spec/jobs/my_job_spec.rb:38:in `block (4 levels) in <top (required)>'
# ./spec/rails_helper.rb:48:in `block (3 levels) in <top (required)>'
# ./spec/rails_helper.rb:47:in `block (2 levels) in <top (required)>'
I've also tried making the job a double and stubbing the retry_on method and that doesn't work either.
I've also tried using Timecop to fast forward the wait time and tests are still failing:
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
expect(AnObject).to receive(:does_something).and_raise { Exception }.exactly(3).times
Timecop.freeze(Time.now + 8.hours) do
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts: 3).exactly(3).times
end
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
It IS a class method of ActiveJob and I've confirmed this in a byebug terminal that this is the case with my job class.
Shouldn't this test work? It's expecting the class to receive the class method with certain arguments. My byebug gets hit when I put it in the retry_on block as well so I know that the method is getting called multiple times.
It's almost as if it's being called on a different class which is very confusing and I don't think is the case but I'm at the end of my rope with this one.
I almost resolved the issue by decoupling my tests from testing the retry_on rails logic itself to testing my business logic around it. This way is better as well in the case that rails ever changes the retry_on logic.
HOWEVER, this does NOT work for more than one test case. If you use this with more than one case, the last test will break and say it has performed more jobs than expected.
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
allow(AnObject).to receive(:does_something).and_raise { Exception }
expect(AnObject).to receive(:does_something).exactly(3).times
expect(Bugsnag).to receive(:notify).with(Exception).once
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
my_job.rb
retry_on Exception, wait: , attempts: 3 do |job, exception|
Bugsnag.notify(exception)
end
def perform(an_object)
an_object.does_something
end
Any help/insight on this would be greatly appreciated.
Would also love a recommendation on how to handle the bubbled up exception after max attempts too. I'm thinking of raising an error within the retry_on block and then have discard_on trigger for the error that's raised.
Thank you wonderful Stack Overflow community!
This is the format of specs needed for retry_on that finally worked for me:
it 'receives retry_on 10 times' do
allow_any_instance_of(MyJob).to receive(:perform).and_raise(MyError.new(nil))
allow_any_instance_of(MyJob).to receive(:executions).and_return(10)
expect(Bugsnag).to receive(:notify)
MyJob.perform_now(an_object)
end
it 'handles error' do
allow_any_instance_of(MyJob).to receive(:perform).and_raise(MyError.new(nil))
expect_any_instance_of(MyJob).to receive(:retry_job)
perform_enqueued_jobs do
MyJob.perform_later(an_object)
end
end
For the first case,
executions is an ActiveJob method that gets run, set and checked every time retry_on is executed. We mock it to return 10 and then expect it to call Bugsnag. retry_on only calls what you gave it in the block once all the attempts have been met. So this works.
For the second case,
Then mock the error to raise for the job instance.
Next we check that it's correctly receiving retry_job (which retry_on calls under the hood) to confirm it's doing the right thing.
Then we wrap the perform_later call in the minitest perform_enqueued_jobs block and call it a day.
The following works fine for me, also for multiple testcases and for testing side effects of the retry_on block.
RSpec.describe MyJob, type: :job do
include ActiveJob::TestHelper
context 'when `MyError` is raised' do
before do
allow_any_instance_of(described_class).to receive(:perform).and_raise(MyError.new)
end
it 'makes 4 attempts' do
assert_performed_jobs 4 do
described_class.perform_later rescue nil
end
end
it 'does something in the `retry_on` block' do
expect(Something).to receive(:something)
perform_enqueued_jobs do
described_class.perform_later rescue nil
end
end
end
end
Note that rescue nil (or some form of rescue) is required if you let exceptions bubble up at the end.
Note that perform_now doesn't count as "enqueued job". So doing described_class.perform_now results in one less attempts counted by assert_performed_jobs.
IMHO you should leave the testing of ActiveJob with the rails team.
You only need to make sure you're configuring the job properly:
it 'retries the job 10 times with 2 minutes intervals' do
allow(MyJob).to receive(:retry_on)
load 'app/path/to/job/my_job.rb'
expect(MyJob).to have_received(:retry_on)
.with(
Exception,
wait: 2.minutes,
attempts: 10
)
end
In the first spec
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts:3).exactly(3).times
This will never gonna work since class method retry_on will be called on class initialization phase, ie when loading that class into memory, not while executing a spec
In the second spec, you tried to make it work using timecop but still failed for the same reason
Third spec is relatively more realistic, but
assert_performed_jobs 3
won't work without passing the block
Something like
assert_performed_jobs 2 do
//call jobs from here
end
While I agree with #fabriciofreitag that one shouldn't need to test the internals of an external library, I think there's definitely value in confirming that your retry_on blocks are configured properly. This setup worked for me without having to worry about how ActiveJob manages the retries.
# app/jobs/my_job.rb
retry_on TimeoutError,
wait: :exponentially_longer,
attempts: 3
# spec/jobs/my_job_spec.rb
describe "error handling" do
before do
ActiveJob::Base.queue_adapter.perform_enqueued_at_jobs = true
end
context "when TimeoutError is raised" do
it "retries timed-out requests" do
expect(client).to receive(:connect).ordered.and_raise(TimeoutError)
expect(client).to receive(:connect).ordered.and_call_original
described_class.perform_now
end
it "retries three times before re-raising" do
expect(client).to receive(:connect)
.exactly(3).times.and_raise(TimeoutError)
expect { described_class.perform_now }.to raise_error(TimeoutError)
end
end
end
I'm using faraday and i have implemented a retry mechanism which is working as expected but i don't understand how can i test it on rspec.
Below is my retry mechanism:
def perform(url:)
max_retries = 3
retry_count = 0
delay = 1
begin
req = config_faraday
response = req.get(url)
JSON.parse(response.body)
rescue Faraday::Error => err
puts "Request failed. Retries left: #{max_retries - retry_count}"
sleep delay += retry_count
retry_count += 1
retry if retry_count < max_retries
ensure
if retry_count == max_retries
raise ApiError.new("Number of retries has been exhausted")
end
end
end
I'm not sure how to handle the case of when api is not working
I have written it by stubbing the get request and making the retry_count to max and then trying to hit the request:
context 'Api is not working' do
let(:retry_count){
3
}
before do
request = double(Faraday)
allow(request).to receive(:get).and_raise(Faraday::Error)
end
it "raise error after retries" do
expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
end
end
But i'm getting an error:
Products::Fetcher Api is not working raise error after retries
Failure/Error: expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
expected Products::ApiError but was not given a block
# ./spec/services/products/fetcher_spec.rb:34:in `block (3 levels) in <top (required)>'
Weird: your tests as such do not test any of the code of the perform. Which is sometimes good, but in this case they do not seem to test anything other than your stub.
For API testing I would suggest looking at a gem like vcr. But this would not work to reproduce the failing, then retrying and then succeeding.
To test that, from the code you showed, I would stub config_faraday to return a mock object, that would raise an error two times when get is called and return a body on the third time, for instance.
Wrong syntax for raise_error matcher
Correct is:
expect { raise StandardError }.to raise_error
Your example:
expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
So correct form should be:
expect{ Products::Fetcher.perform(url: url) }.to raise_error(Products::ApiError)
When debugging failing integration tests, I keep running into the same problem where the exceptions raised in my code are suppressed and not shown in the testing output.
For example, for the following controller and test:
class RegistrationController::ApplicationController
def create
# some code that raises an exception
end
end
class RegistrationFlowTest < ActionDispatch::IntegrationTest
test 'user registers successfully' do
post sign_up_path, params: { username: 'arnold', password: '123' }
assert_response :success
end
end
The output is something like
Minitest::Assertion: Expected response to be a <2XX: success>, but was a <500: Internal Server Error>
Is there a way to see the exact raised exception? Instead of just the difference of HTTP response code?
Thanks!
Simon
My recommended approach to this issue would be to actually parse the response provided by Rails (at least by default in test and development environments) which includes the stacktrace for the error and handle that in the case that your test fails. This has the advantage that it won't output the stacktrace when errors are raised that don't result in failing tests (e.g. scenarios where you are intentionally testing how failures are handled).
This little module that I've crafted will allow you to call assert_response_with_errors to assert the response to a call but output the exception message and stack trace in a readable format when the response is not what you expected.
module ActionDispatch
module Assertions
module CustomResponseAssertions
# Use this method when you want to assert a response body but also print the exception
# and full stack trace to the test console.
# Useful when you are getting errors in integration tests but don't know what they are.
#
# Example:
# user_session.post create_gene_path, params: {...}
# user_session.assert_response_with_errors :created
def assert_response_with_errors(type, message = nil)
assert_response(type, message)
rescue Minitest::Assertion => e
message = e.message
message += "\nException message: #{#response.parsed_body[:exception]}"
stack_trace = #response.parsed_body[:traces][:'Application Trace'].map { |line| line[:trace] }.join "\n"
message += "\nException stack trace start"
message += "\n#{stack_trace}"
message += "\nException stack trace end"
raise Minitest::Assertion, message
end
end
end
end
To use this, you need to include it into ActionDispatch::Assertions before Rails has loaded its stack in your test_helper.rb. So just prepend the include into your test_helper.rb, like this:
ActionDispatch::Assertions.include ActionDispatch::Assertions::CustomResponseAssertions
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
...
This will happen because Rails controllers by default handle exceptions and raise the 500 status, making the exceptions invisible to the test suite (which is very helpful if errors are raised in unit tests of a model). Options for disabling this in your test suite, or alternative workarounds, are discussed here.
The key lines of code from that link, which should be added to test/integration/integration_test_helper.rb:
ActionController::Base.class_eval do
def perform_action
perform_action_without_rescue
end
end
Dispatcher.class_eval do
def self.failsafe_response(output, status, exception = nil)
raise exception
end
end
EDIT: I've noticed that that link is quite old now. I'm really familiar with Rack, so whilst the first block looks ok to me, I'm not sure if the second will still be current. You might need to have a look at the relevant current Rails guide if it needs bringing up to date.
My sidekiq worker use get request to get the speech recognition outcome, the response status will be "SUCCESS" or "FAILED" or "RUNNING". when the status is "RUNNING", I want to retry the sidekiq worker in 10 minutes. How can I retry without sleep or raise a exception. Because sleep will consume too much resource and raise exception will leave newrelic error log which I don't want to record.
class GetAsrTextWorker
include Sidekiq::Worker
sidekiq_options :queue => :default, :retry => 5
sidekiq_retry_in do | count|
600 * (count + 1)
end
def perform(task_id):
# get request to get the outcome
if status == "SUCCESSD"
# save the outcome
elsif status == "FAILED"
raise AsrError
elsif status == "RUNNING"
raise "Asr running"
end
rescue AsrError => e
Sidekiq.logger.error e.message
end
end
This method will retry this worker but will lead to ugly error log in newrelic which I don't want to record.
Use the New Relic Ruby agent option error_collector.ignore_errors to ignore the specific exception you are raising:
Define a custom exception you can raise:
# lib/retry_job.rb
class RetryJob < StandardError; end
Define a worker that raises the exception:
# app/workers/foo_worker.rb
class FooWorker
include Sidekiq::Worker
def perform
raise RetryJob
end
end
Ignore that exception with the New Relic agent:
# config/newrelic.yml
common: &default_settings
error_collector:
ignore_errors: "ActionController::RoutingError,Sinatra::NotFound,RetryJob"