Is it possible, when catching an error in a job, to put this job in dead?
Something like:
class MyJob < ApplicationJob
queue_as :default
sidekiq_options retry: 5
rescue_from MyError do
# This is where I have to put the job in dead.
end
def perform(document)
...
end
end
As per this question ... you cannot dynamically do this within your job. Your best option would be to set the retries to zero.
From the documentation ... Skip retries, send a failed job straight to the Dead set:
class NonRetryableWorker
include Sidekiq::Worker
sidekiq_options retry: 0
def perform(...)
end
end
Related
I am new to Ruby on Rails and Sidekiq. I want to set this delete request to be done in Sidekiq queue and I don't know how to send it to the perform method, I am sending the Book model to the perform method
My controller Action code
def destroy
BaseWorkerJob.perform_async(Book)
end
My BaseWorkerJob class code
class BaseWorkerJob
include Sidekiq::Job
sidekiq_options retry:0
def perform(book)
# Do something
book.find(params[:id]).destroy!
sleep 15
end
end
SideKiq Error
enter image description here
ruby 3.1.2
Rails 7.0.4
You can send the model name and object id to the worker
def destroy
BaseWorkerJob.perform_async(Book.to_s, params[:id])
end
class BaseWorkerJob
include Sidekiq::Job
sidekiq_options retry: 0
def perform(klass_name, object_id)
klass_name.constantize.find(object_id).destroy!
end
end
Try it out!
class User < ApplicationRecord
def update_avatar
#some avatar image processing
end
handle_asynchronously :update_avatar, queue: :image_processing
end
I'm using gem delayed_job_active_record with default config for failed jobs as delete_failed_jobs: true. I would like to not delete jobs on queue image_processing, How can I achieve the case.
As described here, to set a per-job default for destroying failed jobs that overrides the Delayed::Worker.destroy_failed_jobs you can define a destroy_failed_jobs? method on the job
NewsletterJob = Struct.new(:text, :emails) do
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
def destroy_failed_jobs?
false
end
end
in your case something similar to this:
class YourJob
...
def destroy_failed_jobs?
queue_name != 'image_processing'
end
end
I am trying to implement that when inside a worker a model that has paper trail is updated the whodunnit must be set with 'worker', I tried many things without success, the last one was to try a solution found in a github thread the problem is that I have workers with perfom without / with 1 or multiple params( I changed this to set PaperTrail.request.whodunnit = 'worker')
How can I set whodunnit before or around each perform, so that the whodunnit in the versions is saved as whodunnit: worker?
my workers look like this:
module ModuleName
module Workers
class WorkerClass
include Sidekiq::Worker
sidekiq_options queue: 'default', retry: true
def perform # here with none, 1 or more params
# update some model
end
end
end
end
The solution for me
base_worker.rb
class BaseWorker
extend ActiveModel::Callbacks
define_model_callbacks :perform
around_perform :set_paper_trail_whodunnit
def set_paper_trail_whodunnit
PaperTrail.request.whodunnit = "worker-#{self.class.name}"
yield
PaperTrail.request.whodunnit = nil
end
def perform(*args)
run_callbacks(:perform) do
perform!(*args)
end
end
def perform!(*_args)
raise 'Missing your #perform! method implementation'
end
end
workers:
module ModuleName
module Workers
class WorkerClass < ::BaseWorker
include Sidekiq::Worker
sidekiq_options queue: 'default', retry: true
def perform! # here with none, 1 or more params
# update some model
end
end
end
end
I have the following snippet within my job:
before_enqueue do |job|
# do something
#car = create_car
end
before_perform do |job|
# do something
#car.update(type: 'broken')
end
but when the job is performed #car is a nil. Is it possible to pass somehow the instance variable from one callback to the second one? Even only ID would be fine. Cheers.
You would need to make this an instance variable off of job and access that way:
class Car < ActiveJob::Base
attr_accessor :car
end
then
before_enqueue do |job|
# do something
job.car = create_car
end
before_perform do |job|
# do something
job.car.update(type: 'broken')
end
Similar to what you're trying to do, I wanted to know which user enqueued a job without passing the user id to each job in perform. It doesn't look like ActiveJob lets you serialize new instance variables, so I just made use of the arguments variable:
class ApplicationJob < ActiveJob::Base
before_enqueue do |job|
# Get user_id from somewhere first
job.arguments = [user_id, *job.arguments]
end
around_perform do |job, block|
user_id = job.arguments.shift
# Store user_id somewhere
block.call
# Ensure user_id is no longer stored
# (why I'm using around_perform instead of before_perform)
end
end
However, that causes problems if you use perform_now and not just perform_later, since any job performed "now" does not pass through before_enqueue. So here's an improved approach to allow perform_now:
class ApplicationJob < ActiveJob::Base
before_enqueue do |job|
job.arguments = ['_LATER_', user_id, *job.arguments]
end
around_perform do |job, block|
if job.arguments[0] == '_LATER_'
_, user_id = job.arguments.shift(2)
store_somewhere(user_id) { block.call }
else
block.call
end
end
end
There's a controller action in my Rails app that contacts a user via text-message and email. For reasons I won't go into, the text-message needs to complete before the email can be sent successfully. I originally had something like this:
controller:
class MyController < ApplicationController
def contact_user
ContactUserWorker.perform_async(#user.id)
end
end
workers:
class ContactUserWorker
include Sidekiq::Worker
def perform(user_id)
SendUserTextWorker.perform_async(user_id)
SendUserEmailWorker.perform_async(user_id)
end
end
class SendUserTextWorker
include Sidekiq::Worker
def perform(user_id)
user = User.find(user_id)
user.send_text
end
end
class SendUserEmailWorker
include Sidekiq::Worker
def perform(user_id)
user = User.find(user_id)
user.send_email
end
end
This was unreliable; sometimes the email would fail, sometimes both would fail. I'm trying to determine whether perform_async was the cause of the problem. Was the async part allowing the email to fire off before the text had completed? I'm a little fuzzy on how exactly perform_async works, but that sounded like a reasonable guess.
At first, I refactored ContactUserWorker to:
class ContactUserWorker
include Sidekiq::Worker
def perform(user_id)
user = User.find(user_id)
User.send_text
SendUserEmailWorker.perform_async(user_id)
end
end
Eventually though, I just moved the call to send_text out of the workers altogether and into the controller:
class MyController < ApplicationController
def contact_user
#user.send_text
SendUserEmailWorker.perform_async(#user.id)
end
end
This is a simplified version of the real code, but that's the gist of it. It seems to be working fine now, though I still wonder whether the problem was Sidekiq-related or if something else was going on.
I'm curious whether my original structure would've worked if I'd used perform instead of perform_async for all the calls except the email call. Like this:
class MyController < ApplicationController
def contact_user
ContactUserWorker.perform(#user.id)
end
end
class ContactUserWorker
include Sidekiq::Worker
def perform(user_id)
SendUserTextWorker.perform(user_id)
SendUserEmailWorker.perform_async(user_id)
end
end
If the email can only be sent after the text message has been sent, then send the email after successful completion of sending the text.
class ContactUserWorker
include Sidekiq::Worker
def perform(user_id)
SendUserTextWorker.perform_async(user_id)
end
end
class SendUserTextWorker
include Sidekiq::Worker
def perform(user_id)
user = User.find(user_id)
text_sent = user.send_text
SendUserEmailWorker.perform_async(user_id) if text_sent
end
end
class SendUserEmailWorker
include Sidekiq::Worker
def perform(user_id)
user = User.find(user_id)
user.send_email
end
end
In user.send_text you need to handle the fact that neither the text or the email has been sent.
I'm curious whether my original structure would've worked if I'd used perform instead of perform_async for all the calls except the email call
It would have. But this is not what you actually intdending. What you really want is this:
class MyController < ApplicationController
def contact_user
ContactUserWorker.perform_async(#user.id)
end
end
class ContactUserWorker
include Sidekiq::Worker
attr_reader :user_id
def perform(user_id)
#user_id = user_id
user.send_text
user.send_email
end
def user
#user ||= User.find user_id
end
end
The problem was indeed the perform async part. It schedules both tasks to be executed in the background by a separate sidekiq daemon process. i guess your sidekiq is configured to execute the jobs concurrently. In the first version you've first scheduled the ContactUserWorker to perform it's job in a background outside of the current rails request. As this worker is startet later on, it kicks off two separate delayed workers in turn, which are then run in parallel and so there is no way to determine which of the both executes/finishes first.
I don't know what you mean exatly by sending text, but sending an email is an io blocking process and therefore it was a good idea to perform this in a background, because it would be blocking a complete rails process otherwise until the email is delivered (on a typical unicorn/passenger multi-process deployment). And as you actually want to execute both tasks sequentially and as an atomic operation, it's totally fine, performing them by a single sidekiq job/worker.
You also don't have to check if send_text succeeds. Sidekiq will retry the complete job if any part of it fails with an exception