I have been attempting to test this method for the past few days with no luck.
Another thing I'd like to be able to do is rescue the error that bubbles up after the final retry attempt is made.
Please see my comments and code snippets below.
Source code for retry_on is here as well for context.
Here's the sample code and tests:
my_job.rb
retry_on Exception, wait: 2.hours, attempts: 3 do |job, exception|
# some kind of rescue here after job.exceptions == 3
# then notify Bugsnag of failed final attempt.
end
def perform(an_object)
an_object.does_something
end
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
expect(AnObject).to receive(:does_something).and_raise { Exception }.exactly(3).times
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts: 3).exactly(3).times
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
The test failure response:
1) MyJob.perform receives retry_on 3 times
Failure/Error: expect(job).to receive(:retry_on).with(wait: 4.hours, attempts: 3).exactly(3).times
(MyJob (class)).retry_on({:wait=>2 hours, :attempts=>3})
expected: 3 times with arguments: ({:wait=>2 hours, :attempts=>3})
received: 0 times
# ./spec/jobs/my_job_spec.rb:38:in `block (4 levels) in <top (required)>'
# ./spec/rails_helper.rb:48:in `block (3 levels) in <top (required)>'
# ./spec/rails_helper.rb:47:in `block (2 levels) in <top (required)>'
I've also tried making the job a double and stubbing the retry_on method and that doesn't work either.
I've also tried using Timecop to fast forward the wait time and tests are still failing:
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
expect(AnObject).to receive(:does_something).and_raise { Exception }.exactly(3).times
Timecop.freeze(Time.now + 8.hours) do
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts: 3).exactly(3).times
end
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
It IS a class method of ActiveJob and I've confirmed this in a byebug terminal that this is the case with my job class.
Shouldn't this test work? It's expecting the class to receive the class method with certain arguments. My byebug gets hit when I put it in the retry_on block as well so I know that the method is getting called multiple times.
It's almost as if it's being called on a different class which is very confusing and I don't think is the case but I'm at the end of my rope with this one.
I almost resolved the issue by decoupling my tests from testing the retry_on rails logic itself to testing my business logic around it. This way is better as well in the case that rails ever changes the retry_on logic.
HOWEVER, this does NOT work for more than one test case. If you use this with more than one case, the last test will break and say it has performed more jobs than expected.
my_spec.rb
it 'receives retry_on 3 times' do
perform_enqueued_jobs do
allow(AnObject).to receive(:does_something).and_raise { Exception }
expect(AnObject).to receive(:does_something).exactly(3).times
expect(Bugsnag).to receive(:notify).with(Exception).once
MyJob.perform_later(an_object)
end
assert_performed_jobs 3
end
my_job.rb
retry_on Exception, wait: , attempts: 3 do |job, exception|
Bugsnag.notify(exception)
end
def perform(an_object)
an_object.does_something
end
Any help/insight on this would be greatly appreciated.
Would also love a recommendation on how to handle the bubbled up exception after max attempts too. I'm thinking of raising an error within the retry_on block and then have discard_on trigger for the error that's raised.
Thank you wonderful Stack Overflow community!
This is the format of specs needed for retry_on that finally worked for me:
it 'receives retry_on 10 times' do
allow_any_instance_of(MyJob).to receive(:perform).and_raise(MyError.new(nil))
allow_any_instance_of(MyJob).to receive(:executions).and_return(10)
expect(Bugsnag).to receive(:notify)
MyJob.perform_now(an_object)
end
it 'handles error' do
allow_any_instance_of(MyJob).to receive(:perform).and_raise(MyError.new(nil))
expect_any_instance_of(MyJob).to receive(:retry_job)
perform_enqueued_jobs do
MyJob.perform_later(an_object)
end
end
For the first case,
executions is an ActiveJob method that gets run, set and checked every time retry_on is executed. We mock it to return 10 and then expect it to call Bugsnag. retry_on only calls what you gave it in the block once all the attempts have been met. So this works.
For the second case,
Then mock the error to raise for the job instance.
Next we check that it's correctly receiving retry_job (which retry_on calls under the hood) to confirm it's doing the right thing.
Then we wrap the perform_later call in the minitest perform_enqueued_jobs block and call it a day.
The following works fine for me, also for multiple testcases and for testing side effects of the retry_on block.
RSpec.describe MyJob, type: :job do
include ActiveJob::TestHelper
context 'when `MyError` is raised' do
before do
allow_any_instance_of(described_class).to receive(:perform).and_raise(MyError.new)
end
it 'makes 4 attempts' do
assert_performed_jobs 4 do
described_class.perform_later rescue nil
end
end
it 'does something in the `retry_on` block' do
expect(Something).to receive(:something)
perform_enqueued_jobs do
described_class.perform_later rescue nil
end
end
end
end
Note that rescue nil (or some form of rescue) is required if you let exceptions bubble up at the end.
Note that perform_now doesn't count as "enqueued job". So doing described_class.perform_now results in one less attempts counted by assert_performed_jobs.
IMHO you should leave the testing of ActiveJob with the rails team.
You only need to make sure you're configuring the job properly:
it 'retries the job 10 times with 2 minutes intervals' do
allow(MyJob).to receive(:retry_on)
load 'app/path/to/job/my_job.rb'
expect(MyJob).to have_received(:retry_on)
.with(
Exception,
wait: 2.minutes,
attempts: 10
)
end
In the first spec
expect(MyJob).to receive(:retry_on).with(wait: 2.hours, attempts:3).exactly(3).times
This will never gonna work since class method retry_on will be called on class initialization phase, ie when loading that class into memory, not while executing a spec
In the second spec, you tried to make it work using timecop but still failed for the same reason
Third spec is relatively more realistic, but
assert_performed_jobs 3
won't work without passing the block
Something like
assert_performed_jobs 2 do
//call jobs from here
end
While I agree with #fabriciofreitag that one shouldn't need to test the internals of an external library, I think there's definitely value in confirming that your retry_on blocks are configured properly. This setup worked for me without having to worry about how ActiveJob manages the retries.
# app/jobs/my_job.rb
retry_on TimeoutError,
wait: :exponentially_longer,
attempts: 3
# spec/jobs/my_job_spec.rb
describe "error handling" do
before do
ActiveJob::Base.queue_adapter.perform_enqueued_at_jobs = true
end
context "when TimeoutError is raised" do
it "retries timed-out requests" do
expect(client).to receive(:connect).ordered.and_raise(TimeoutError)
expect(client).to receive(:connect).ordered.and_call_original
described_class.perform_now
end
it "retries three times before re-raising" do
expect(client).to receive(:connect)
.exactly(3).times.and_raise(TimeoutError)
expect { described_class.perform_now }.to raise_error(TimeoutError)
end
end
end
Related
Context:
I have a job that catch some errors with these configuration
class CheckBankStatusJob < BaseJob
discard_on(ConnectionErrors::Error) do
Logger.new(STDOUT).warn("Connection Error!")
end
retry_on(
BankError::NoResponseData,
StandardError,
wait: 5.seconds,
)
end
ConnectionErrors inherit from StandardError
BankError::NoResponseData inherit from StandardError
i want to test the discard_on event in rspec
it "logs down when RemoteErrors::Error" do
allow_any_instance_of(described_class).to receive(:perform).and_raise(ConnectionErrors::Error)
expect_any_instance_of(Logger).to receive(:warn).with("Connection Error!")
described_class.perform_now
end
it never go to discard_on block, instead it goes to retry_on block.
errors
Failure/Error:
expected: 1 time with arguments: xxx
received: 0 times
Does retry_on get executed first? or it is because the ConnectionErrors get inherited from StandardError?
thanks in advance
I think you need to replace "perform" with "perform_now" to raise ConnectionErrors::Error
allow_any_instance_of(described_class).to receive(:perform_now).and_raise(ConnectionErrors::Error)
I am trying to test sending an email with Sidekiq and I am having an error which means it doesn't enqueue since my test shows it didn't change from 0 to 1 in size. Could it be I am missing something out or how can I fix this?
Error
expected `Array#size` to have changed by 1, but was changed by 0
0) Sending Mail sends email to sidekiq
Failure/Error:
expect do
OrderPointsMailer.paid_order_email(customer_detail.id).deliver_later
end.to change(Sidekiq::Worker.jobs, :size).by(1)
expected `Array#size` to have changed by 1, but was changed by 0
# ./spec/requests/order_points_mailer_spec.rb:9:in `block (2 levels) in <top (required)>'
1 example, 1 failure, 0 passed
binding.pry
This shows Sidekiq::Worker.jobs is empty array and it is understandable why it failed but I do not know how I could fix this.
[1] pry(#<RSpec::ExampleGroups::SendingMailWithSidekiq>)>Sidekiq::Worker.jobs
=> []
app/jobs/order_points_job.rb
# frozen_string_literal: true
class OrderPointsJob < ApplicationJob
def perform(customer_email)
customer_detail = CustomerDetail.find_by(email: customer_email)
return unless customer_detail
OrderPointsMailer.paid_order_email(customer_detail.id).deliver_later # This call send email to sidekiq process
end
end
RSpec
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe 'Sending Mail with Sidekiq', type: :request do
it 'sends email to sidekiq' do
customer_detail = create(:customer_detail)
expect do
OrderPointsMailer.paid_order_email(customer_detail.id).deliver_later
end.to change(Sidekiq::Worker.jobs, :size).by(1)
end
end
rails_helper.rb
require 'sidekiq/testing'
Sidekiq::Testing.fake!
You probably should use the method count instead of size:
.to change(Sidekiq::Worker.jobs.reload, :count).by(1)
Take a look at the size implementation:
From: /home/toptal/.rvm/gems/ruby-2.5.5/gems/activerecord-6.0.0/lib/active_record/relation.rb # line 260:
Owner: ActiveRecord::Relation
Visibility: public
Number of lines: 3
def size
loaded? ? #records.length : count(:all)
end
Also take a look on the matcher implementation (https://github.com/rspec/rspec-expectations/blob/master/lib/rspec/matchers/built_in/change.rb#L82:L85)
I'm guessing that the receiver (Sidekiq::Worker.jobs in your case) will be evaluated once, and then the size would be called twice on the same objects collection, without reloading.
One other approach you could take would be
.to change { Sidekiq::Worker.jobs.reload.size }.by(1)
But this would be a bit wasteful in case you have more records in the DB, because each time the AR objects will need to be created (and they're not used)
I'm using faraday and i have implemented a retry mechanism which is working as expected but i don't understand how can i test it on rspec.
Below is my retry mechanism:
def perform(url:)
max_retries = 3
retry_count = 0
delay = 1
begin
req = config_faraday
response = req.get(url)
JSON.parse(response.body)
rescue Faraday::Error => err
puts "Request failed. Retries left: #{max_retries - retry_count}"
sleep delay += retry_count
retry_count += 1
retry if retry_count < max_retries
ensure
if retry_count == max_retries
raise ApiError.new("Number of retries has been exhausted")
end
end
end
I'm not sure how to handle the case of when api is not working
I have written it by stubbing the get request and making the retry_count to max and then trying to hit the request:
context 'Api is not working' do
let(:retry_count){
3
}
before do
request = double(Faraday)
allow(request).to receive(:get).and_raise(Faraday::Error)
end
it "raise error after retries" do
expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
end
end
But i'm getting an error:
Products::Fetcher Api is not working raise error after retries
Failure/Error: expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
expected Products::ApiError but was not given a block
# ./spec/services/products/fetcher_spec.rb:34:in `block (3 levels) in <top (required)>'
Weird: your tests as such do not test any of the code of the perform. Which is sometimes good, but in this case they do not seem to test anything other than your stub.
For API testing I would suggest looking at a gem like vcr. But this would not work to reproduce the failing, then retrying and then succeeding.
To test that, from the code you showed, I would stub config_faraday to return a mock object, that would raise an error two times when get is called and return a body on the third time, for instance.
Wrong syntax for raise_error matcher
Correct is:
expect { raise StandardError }.to raise_error
Your example:
expect(Products::Fetcher.perform(url: url)).to raise_error(Products::ApiError)
So correct form should be:
expect{ Products::Fetcher.perform(url: url) }.to raise_error(Products::ApiError)
When debugging failing integration tests, I keep running into the same problem where the exceptions raised in my code are suppressed and not shown in the testing output.
For example, for the following controller and test:
class RegistrationController::ApplicationController
def create
# some code that raises an exception
end
end
class RegistrationFlowTest < ActionDispatch::IntegrationTest
test 'user registers successfully' do
post sign_up_path, params: { username: 'arnold', password: '123' }
assert_response :success
end
end
The output is something like
Minitest::Assertion: Expected response to be a <2XX: success>, but was a <500: Internal Server Error>
Is there a way to see the exact raised exception? Instead of just the difference of HTTP response code?
Thanks!
Simon
My recommended approach to this issue would be to actually parse the response provided by Rails (at least by default in test and development environments) which includes the stacktrace for the error and handle that in the case that your test fails. This has the advantage that it won't output the stacktrace when errors are raised that don't result in failing tests (e.g. scenarios where you are intentionally testing how failures are handled).
This little module that I've crafted will allow you to call assert_response_with_errors to assert the response to a call but output the exception message and stack trace in a readable format when the response is not what you expected.
module ActionDispatch
module Assertions
module CustomResponseAssertions
# Use this method when you want to assert a response body but also print the exception
# and full stack trace to the test console.
# Useful when you are getting errors in integration tests but don't know what they are.
#
# Example:
# user_session.post create_gene_path, params: {...}
# user_session.assert_response_with_errors :created
def assert_response_with_errors(type, message = nil)
assert_response(type, message)
rescue Minitest::Assertion => e
message = e.message
message += "\nException message: #{#response.parsed_body[:exception]}"
stack_trace = #response.parsed_body[:traces][:'Application Trace'].map { |line| line[:trace] }.join "\n"
message += "\nException stack trace start"
message += "\n#{stack_trace}"
message += "\nException stack trace end"
raise Minitest::Assertion, message
end
end
end
end
To use this, you need to include it into ActionDispatch::Assertions before Rails has loaded its stack in your test_helper.rb. So just prepend the include into your test_helper.rb, like this:
ActionDispatch::Assertions.include ActionDispatch::Assertions::CustomResponseAssertions
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
...
This will happen because Rails controllers by default handle exceptions and raise the 500 status, making the exceptions invisible to the test suite (which is very helpful if errors are raised in unit tests of a model). Options for disabling this in your test suite, or alternative workarounds, are discussed here.
The key lines of code from that link, which should be added to test/integration/integration_test_helper.rb:
ActionController::Base.class_eval do
def perform_action
perform_action_without_rescue
end
end
Dispatcher.class_eval do
def self.failsafe_response(output, status, exception = nil)
raise exception
end
end
EDIT: I've noticed that that link is quite old now. I'm really familiar with Rack, so whilst the first block looks ok to me, I'm not sure if the second will still be current. You might need to have a look at the relevant current Rails guide if it needs bringing up to date.
I'm trying to write some tests involving file operations. I want to use some fake file system (something like VCR for external services) and I have found fakeFS. Unfortunately, either I can't set it right or something is broken (which I doubt, it's quite basic function), I've prepared simple example which illustrates what I mean, let the code speak:
With real FS:
module MyModule
describe Something do
before(:all) do
File.open("#{Rails.root}/foo.txt", 'w+') { |f| f.write 'content'}
end
it 'should exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
it 'should still exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
end
end
Running that gives:
bash-4.2$ rspec
..
Finished in 0.00161 seconds
2 examples, 0 failures
Adding fakeFS in such way:
require 'fakefs/spec_helpers'
module MyModule
describe Something do
include FakeFS::SpecHelpers
FakeFS.activate!
FakeFS::FileSystem.clone(Rails.root)
before(:all) do
File.open("#{Rails.root}/foo.txt", 'w+') { |f| f.write 'content'}
end
it 'should exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
it 'should still exist' do
expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
end
end
end
results in:
bash-4.2$ rspec
.F
Failures:
1) MyModule::Something should still exist
Failure/Error: expect(Pathname.new("#{Rails.root}/foo.txt").exist?).to be_true
expected: true value
got: false
# ./spec/models/something_spec.rb:23:in `block (2 levels) in <module:MyModule>'
Finished in 0.00354 seconds
2 examples, 1 failure
So it seems like file is not persisted through subsequent tests. Do I misunderstand how before(:all) works or do I do something wrong? If so then why that code works with real files?
If it is 'not a bug, just a feature' then is there any other fake filesystem gem which is consistent with real one? Or do I have to stay with real files to get tests that.. well, test?
I found the answer just after creating that question, duh ;) I've looked into source of that lib and found suspicious line.
Instead of FakeFS::SpecHelpers I've included FakeFS::SpecHelpers::All which is the same code except FakeFS::FileSystem is not being cleared after each call, now it behaves correctly.