I would like Airbrake to only be notified of errors when the retries are exhausted, but I can't seem to think of a way to implement it...
I can add a sidekiq_retries_exhausted hook to send the error to AirBrake but the only way I can think of catching the actual failures is to add a middleware that swallows the error, but then, the job will be marked as a success if there is no error... then there will never be any retries..
Hope that makes sense!
I managed to implement this with a Sidekiq middleware that is inserted at the start of the list:
class RaiseOnRetriesExtinguishedMiddleware
include Sidekiq::Util
def call(worker, msg, queue)
yield
rescue Exception => e
bubble_exception(msg, e)
end
private
def bubble_exception(msg, e)
max_retries = msg['retries'] || Sidekiq::Middleware::Server::RetryJobs::DEFAULT_MAX_RETRY_ATTEMPTS
retry_count = msg['retry_count'] || 0
last_try = !msg['retry'] || retry_count == max_retries - 1
raise e if last_try
end
def retry_middleware
#retry_middleware ||= Sidekiq::Middleware::Server::RetryJobs.new
end
end
If its the last try and its thrown an exception, it'll let it bubble up (to Airbrake) otherwise it won't. This doesn't affect failure recording as that happens later in the chain.
As shown here (not my code):
Airbrake.configure do |config|
config.api_key = '...'
config.ignore_by_filter do |exception_data|
exception_data[:parameters] &&
exception_data[:parameters]['retry_count'].to_i > 0
end
end
I ran into the exact same thing, and wanted to keep it out of AirBrake. Here is what I did, which is easy to read and simple:
class TaskWorker
include Sidekiq::Worker
class RetryLaterNotAnError < RuntimeError
end
def perform task_id
task = Task.find(task_id)
task.do_cool_stuff
if task.finished?
#log.debug "Nothing to do for task #{task_id}"
return false
else
raise RetryLaterNotAnError, task_id
end
end
end
And then, to get Airbrake to ignore it:
Airbrake.configure do |config|
config.ignore << 'RetryLaterNotAnError'
end
Voila!
Here is how we do it for Bugsnag, which you can customise for Airbrake.
# config/initializers/00_core_ext.rb
class StandardError
def skip_bugsnag?
!!#skip_bugsnag
end
def skip_bugsnag!
#skip_bugsnag = true
return self
end
end
# config/initializers/bugsnag.rb
config.ignore_classes << lambda { |e| e.respond_to?(:skip_bugsnag?) && e.skip_bugsnag? }
# In Sidekiq Jobs
raise ErrorToRetryButNotReport.new("some message").skip_bugsnag!
# Or if the error is raised by a third party
begin
# some code that calls a third-party method
rescue ErrorToRetryButNotReport => e
e.skip_bugsnag!
raise
end
You can then manually choose to send the error from sidekiq_retries_exhausted.
Related
I'd like to retry a function with different params depending on the result of the first iteration:
Giving a retry function like follow:
def retry_on_fail(**args)
yield
rescue StandardError => e
args = args.merge(different_param => true) if e.class == `specific_error`
retry
Is there a way to do so? I didn't find it yet...
Thanks!
You can yield however many times you want in a method and the trick is really passing the arguments to the block:
# given
class SpecificError < StandardError; end
def retry_on_fail(**args)
yield(args)
rescue SpecificError
yield(args.merge(different_param: true))
end
retry_on_fail do |args|
raise SpecificError if args.empty?
args
end
# returns { different_param: true }
There is also a slight differnce here flow wise - retry runs the whole method from the top and this will just call the block again. If thats what you want you could do:
def retry_on_fail(**args)
yield(args)
rescue SpecificError
args.merge!(different_param: true)
retry
end
But this has the potential to create an endless loop if the block raises the same exception again.
Try this
def retry_on_fail(**args)
rescue_args = args
begin
yield(rescue_args)
rescue StandardError => e
rescue_args = rescue_args.merge(different_param => true) if e.class == `specific_error`
retry
end
end
I have a model, Transaction, and a method, external_evaluation. external_evaluation works its way down the stack and eventually calls out to an out to an AWS lambda. When the response is bad, a BadResponse exception is raised.
There is a pattern in the codebase that gets used frequently that goes something like
def get_some_transactions()
Transaction.where(some_column: some_expression)
end
def do_some_stuff()
get_some_transactions.each do |transaction|
do_something(transaction.external_evaluation)
rescue BadResponse => e
log(e)
next
end
end
def do_some_other_stuff()
get_some_transactions.each_with_object({}) do |transaction, transaction_hash|
transaction_hash[transaction] = do_something_else(transaction.external_evaluation)
rescue BadResponse => e
log(e)
next
end
end
I really dislike the duplication of the error handling code in this pattern, and would like to be able to add default error handling into get_some_transactions which will apply regardless of which iteration function is called (each, each_with_object, each_with_index, ...). Is there an idiomatic way to do this in Ruby?
def with_error_handing(&block)
begin
yield
rescue BadResponse => e
log(e)
end
end
def do_some_stuff()
get_some_transactions.each do |transaction|
with_error_handing do
do_something(transaction.external_evaluation)
end
end
end
def do_some_other_stuff()
get_some_transactions.each_with_object({}) do |transaction, transaction_hash|
with_error_handing do
transaction_hash[transaction] = do_something_else(transaction.external_evaluation)
end
end
end
You can move the rescue to external_evaluation method.
ActiveRecord::RecordNotFound in Sidekiq worker when I save object. I don't use rails callbacks.
I run worker from service, and save object in this service.
class LeadSmsSendingService < Rectify::Command
...initialize params
def send_sms_message
sms_conversation = lead.sms_conversations.find_or_create_by(sms_number: sms_number)
attrs = sms_form.to_hash.symbolize_keys.slice(:body, :direction, :from, :to)
.merge(campaign_id: campaign_id)
sms_message = sms_conversation.sms_messages.build(attrs)
sms_message.to ||= lead.phone
sms_message.body = VariableReplacement.new(lead).render(sms_message.body)
# #todo we need to raise an exception here
return unless sms_message.save
DeliverSmsMessageWorker.perform_in(3.seconds, sms_message.id, 'LeadSmsSendingService')
end
end
class DeliverSmsMessageWorker
include Sidekiq::Worker
sidekiq_options queue: 'priority'
def perform(sms_message_id, from_where="Unknown")
sms_message = SmsMessage.find(sms_message_id)
sms_message.deliver!
rescue StandardError => e
Bugsnag.notify(e) do |report|
# Add information to this report
report.add_tab(:worker, { from_where: from_where.to_s })
end
end
end
Seems that the record has still to be commited, even if it sounds strange because of the 3 seconds delay. Does it work if you increase this delay?
This link could be useful: https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting#cannot-find-modelname-with-id12345
Is there a way to find out whether any exception was raised and rescued during the execution of some code?
Preferably in tests written with ActiveSupport::TestCase and not RSpec
Is there any global ruby exception stack or something, which I could check?
If you want to be a monster, you can instrument the errors themselves:
class StandardError
##called = false
def initialize
##called = true
super
end
def self.called
##called
end
end
#test it out like so:
def raise_arg_error
raise ArgumentError
rescue
end
puts ArgumentError.called #false
raise_arg_error
puts ArgumentError.called #true
Great for ad hoc sanity checks. Terrible for production code.
As clarified in the comments, OP needed it for debug purposes as opposed to write tests with it.
Kernel#set_trace_func lets you intercept low level events such as an error being raised:
set_trace_func(proc do |event, *_|
puts 'Raised!' if event == 'raise'
end)
raise 'Oh, no!' rescue :foo
You can run #set_trace_func before the code you were trying to debug. If no exception was raised, but a raise was registered by the hook - someone rescued it.
This can create some noise depending on what you are executing. Fortunately, you can filter it down by file and line:
set_trace_func(proc do |event, file, line, *_|
if event == 'raise' && file == 'test.rb' && line == 42
puts "Raised on #{file}:#{line}!"
end
end)
It even gives you the binding, so you can drop down a debugger:
require 'irb'
set_trace_func(proc do |event, *_, error_binding, _|
error_binding.irb if event == 'raise'
end)
def foo
bar = 42
raise 'Oh, no!' rescue :baz
end
foo
# Opens an irb
# > bar # => 42
My assertion example is below,
class test < Test::Unit::TestCase
def test_users
begin
assert_equal(user.name, 'John')
assert_equal(user.age, 30)
assert_equal(user.zipcode, 500002)
rescue Exception
raise
end
end
end
If any one of assertions fails, i should move on to process the next one and collect the failure assertions and show failures the end of the result.
I have used add_failure method, its working for looping condition
rescue Test::Unit::AssertionFailedError => e
add_failure(e.message, e.backtrace)
Can any one help ?
A good unit test should test exactly one thing, specifically to avoid problems like you just face. This test case will report on all failed tests, and not just the first failed test:
class MyTest < Test::Unit::TestCase
def test_user_name
assert_equal(user.name, 'John')
end
def test_user_age
assert_equal(user.age, 30)
end
def test_user_zipcode
assert_equal(user.zipcode, 500002)
end
end
Your main problem is that assert_equal ends up calling assert (as shown below) and assert will raise an ArgumentException.
File test/unit/assertions.rb, line 144
def assert_equal(exp, act, msg = nil)
msg = message(msg) {
# omitted code to save space
}
assert(exp == act, msg)
end
File test/unit/assertions.rb, line 29
def assert(test, msg = UNASSIGNED)
case msg
when UNASSIGNED
msg = nil
when String, Proc
else
bt = caller.reject { |s| s.rindex(MINI_DIR, 0) }
raise ArgumentError, "assertion message must be String or Proc, but #{msg.class} was given.", bt
end
super
end
You could extend Test::Unit::Assertions and provide an assertion that does not raise ArgumentError, which is what is stopping continuation past the failed assertion.
See this question for advice on going that direction and adding in safe assertions.
Please find this code for Continue assertion after failures in Ruby :
def raise_and_rescue
begin
puts 'I am before the raise.'
raise 'An error has occured.'
puts 'I am after the raise.'
rescue
puts 'I am rescued.'
end
puts 'I am after the begin block.'
end
Output :
ruby p045handexcp.rb
I am before the raise.
I am rescued.
I am after the begin block.
Exit code: 0