Can I use the hooks of a delayed job to stop it before running?
If so how?
class Jobs::SomeJob < Struct.new
def before(job)
if should_not_run_job
# Do I raise an exception?
# Is there an official way to stop a Job from running?
end
end
def perform
# Working...
end
def queue_name
return "SomeJob";
end
end
Do I raise an error?
Which hook is recommended to use?
Raising an exception wouldn't work because then the retry mechanism would kick in and would retry running the job a couple of times.
Instead, I would just add a guard clause that in the first line of the perform method that returns without doing anything when the conditions is not true anymore. It depends on how your condition looks like but something like this might work for you:
def perform
return if job_should_not_run_anymore?
# Working
end
private
def job_should_not_run_anymore?
# Condition
end
When your condition is based on the job itself that is only available in a hook method like before but not in the perform method then I would store the result of the condition in an instance variable and check that variable in the perform method like this:
def before(job)
#outdated = job.run_at > 15.minutes.ago
end
def perform
return if #outdated
# Working
end
Related
I am using the most basic version of delayed_job in a Rails app. I have the max time allowed for a delayed_job set at 10 minutes. I would like to get the hooks/callbacks working so I can do something after a job stop executing at the 10 minute mark.
I have this set in my rails app:
config.active_job.queue_adapter = :delayed_job
This is how I normally queue a job:
object.delay.object_action
The hook/callback example is for a named job but the basic, getting started steps are not for a named job. So I don't think I have a named job. Here is the example given to get the callbacks working:
class ParanoidNewsletterJob < NewsletterJob
def enqueue(job)
record_stat 'newsletter_job/enqueue'
end
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
def before(job)
record_stat 'newsletter_job/start'
end
def after(job)
record_stat 'newsletter_job/after'
end
def success(job)
record_stat 'newsletter_job/success'
end
def error(job, exception)
Airbrake.notify(exception)
end
def failure(job)
page_sysadmin_in_the_middle_of_the_night
end
end
I would love to get the after or error hooks/callbacks to fire.
Where do I put these callbacks in my Rails app to have them fire for the basic delayed_job setup? If I should be using ActiveJob callbacks where do you put those callbacks given delayed_job is being used?
You cannot use object.delay.object_action convenience syntax if you want more advanced features like callbacks. The #delay convenience method will generate a job object that works similar to this:
# something like this is already defined in delayed_job
class MethodCallerJob
def initialize(object, method, *args)
#object = object
#method = method
#args = args
end
def perform
#object.send(#method, *#args)
end
end
# `object.delay.object_action` does the below automatically for you
# instantiates a job with your object and method call
job = MethodCallerJob.new(object, :object_action, [])
Delayed::Job.enqueue(job) # enqueues it for running later
then later, in the job worker, something like the below happens:
job = Delayed::Job.find(job_id) # whatever the id turned out to be
job.invoke_job # does all the things, including calling #perform and run any hooks
job.delete # if it was successful
You have to create what the delayed_job README calls "Custom Jobs", which are just plain POROs that have #perform defined at a minimum. Then you can customize it and add all the extra methods that delayed_job uses for extra features like max_run_time, queue_name, and the ones you want to use: callbacks & hooks.
Sidenote: The above info is for using delayed_job directly. All of the above is possible using ActiveJob as well. You just have to do it the ActiveJob way by reading the documentation & guides on how, just as I've linked you to the delayed_job README, above.
You can create delayed_job hooks/callback by something like this
module Delayed
module Plugins
class TestHooks < Delayed::Plugin
callbacks do |lifecycle|
lifecycle.before(:perform) do |_worker, job|
.....
end
end
end
end
end
And need this plugin to initializer
config/initializers/delayed_job.rb
require_relative 'path_to_test_plugin'
Delayed::Worker.plugins << Delayed::Plugins::TestHooks
Similar to perform there are also hooks for success failure and error.
And similar to 'before' you can also capture the 'after' hooks.
I have a job created with rails g job cleanUp.
Is where any option to check if Job is running? Something like this CleanUpJob.isRunning?
If where is no way to make it without additional gems, which will be the simplest? delayed_job?
Second thing to control Job is progress, any thoughts how to implement CleanUpJob.progress or progress_job should be my choice?
briefly:
I need to create a job with two methods (isRunning?, progress).
I don't really want additional tables if possible.
You can use Class: Sidekiq::ScheduledSet for this purpose.
Documentation here
This Class is used in Sidekiq web interface.
Example:
Save job id (jid) when set job. Then you can call it for queried instance
def is_running?
require 'sidekiq/api'
ss = Sidekiq::ScheduledSet.new
jobs = ss.select {|ret| ret.jid == self.jid}
jobs.any?
end
Or, you can set DB flag inside Job with around_perform hook.
class SomeJob < ActiveJob::Base
queue_as :some_job
around_perform do |job, block|
start_import_process_log job.arguments[0], job.arguments[1] || {}
block.call
finish_import_process_log
end
# ...
private
def start_import_process_log import_process, options={}
#some actions
end
def finish_import_process_log
end
end
In this example associated log record is created.
Or you can use before_perform/ after_perform.
In my practice I'm using creting log records on long tasks.
When I need to find and kill job as example - I'm using Sidekiq::ScheduledSet.
I'm trying to run all callback methods manually inside a method. For example, I want to run all "before_destroy" methods inside my model.
Is there a way to manually trigger this? For example, something like:
def some_method
# ...
trigger(:before_destroy)
end
which will then run all methods that I have declared with "before_destroy :...."
Any ideas?
If you're happy to run both :before and :after hooks, you can try run_callbacks.
From the docs:
run_callbacks(kind, &block)
Runs the callbacks for the given event.
Calls the before and around callbacks in the order they were set, yields the block (if given one), and then runs the after callbacks in reverse order.
If the callback chain was halted, returns false. Otherwise returns the result of the block, or true if no block is given.
run_callbacks :save do
save
end
class Foo < ActiveRecord::Base
def destroy_method_1
end
def destroy_method_2
end
before_destroy :destroy_method_1, :destroy_method_2
DESTROY_METHODS = [:destroy_method_1, :destroy_method_2]
def some_method
DESTROY_METHODS.each {|m| send(m) }
end
end
I want the execution of a method in a controller to run in background, so that the speed is not affected. I found out that this could be done using delayed jobs.
following is the method I want to delay:
private
def update_product_count(skus, qty)
path = File.expand_path('../../../voylla_scripts/eBay', __FILE__)
system "python2 "+path+"/ReviseOnOrder.py #{skus.to_json} #{qty.to_json} #{path}> output"
end
I tried using:
def show
if defined? Delayed::Job
Delayed::Job.enqueue(update_product_count(#skus.to_s, #qty.to_s))
end
end
This runs the script within the delayed method, but gives error:
ArgumentError in OrdersController#show
Cannot enqueue items which do not respond to perform
and the view corresponding to the show does not get rendered.
then I tried:
def show
delay.update_product_count(#skus.to_s, #qty.to_s)
end
This doesn't run the method and also gives the following error:
ArgumentError in OrdersController#show
wrong number of arguments (1 for 0)
I also tried handle_asynchronously :update_product_count. But this too gives wrong number of arguments (1 for 0)
Could someone please help me figure this out. Thanks
EDIT: the following change does not give any error, but the script does seem to run
/lib/update_count.rb
class UpdateCount < Struct.new(:skus, :qty, :path)
def perform
system "python2 "+path+"/ReviseOnOrder.py #{skus.to_json} #{qty.to_json} #{path}"
end
end
/app/controller/order_controller.rb:
require 'update_count'
def show
Delayed::Job.enqueue(UpdateCount.new(#skus.to_s, #qty.to_s, path))
end
Place the code you want to execute in perform method, and enqueu the class in the delayed jobs, which when executed will call the perform method
Eg:
/lib/product_count.rb
class ProductCount < Struct.new(:skus, :qty)
def perform
path = File.expand_path('../../../voylla_scripts/eBay', __FILE__)
system "python2 "+path+"/ReviseOnOrder.py #{skus.to_json} #{qty.to_json} #{path}> output"
end
end
Call the delayed job
Delayed::Job.enqueue(ProductCount.new(#skus.to_s, #qty.to_s), :queue => "product_count")
You need to create a new class with public method "perform" which will incapsulate all job that you want:
class MegaJob
def initialize(options=nil)
#skus = options[:skus]
#qty = options[:qty]
end
def perform
update_product_count
end
private
def update_product_count
path = File.expand_path('../../../voylla_scripts/eBay', __FILE__)
system "python2 "+path+"/ReviseOnOrder.py #{#skus.to_json} #{#qty.to_json} #{path}> output"
end
end
To start this job:
Delayed::Job.enqueue MegaJob.new(skus: your_skus, qty: your_qty)
PS Don`t copy and paste the example!
Kinda embarrassing:
script/delayed_job start.
phew!!
i'm trying to get the run_at datetime in a custom job class. my code looks like this:
class MyCustomJob < Struct.new(:my_object)
def perform
if self.run_at == my_object.start_time
# process the job
end
end
end
i also tried a Delayed::Job.find(self) but had no luck.
thanks in advance
If you define a before method on your custom job, the worker will pass you the delayed job instance before calling perform:
class MyCustomTask
def before(job)
#job = job
end
def perform
# has access to #job object.
# You may need to call #job.reload to see in-flight changes to object in the database.
end
end
You should handle this when you create the job: priority = 0
run_time = my_object.start_time
Delayed::Job.enqueue(MyCustomJob.new(my_object), priority, run_time)
https://github.com/tobi/delayed_job/wiki
If your jobs aren't running at the expected time, you may be scheduling them for UTC:
http://www.gregbenedict.com/2009/08/19/is-delayed-job-run_at-datetime-giving-you-fits/
To check the queue for an existing job - you could do the following:
class MyCustomJob < Struct.new(:object_id)
def self.exists?(object_id)
Delayed::Job.where(['handler = ? and failed_at is null',handler(object_id)]).count(:all) > 0
end
def self.handler(object_id)
"--- !ruby/struct:MyCustomJob \nobject_id: #{object_id}\n"
end
def perform
my_object = MyObject.find(object_id)
my_object.do_stuff
end
end
Then just check for MyCustomJob.exists?(my_object.id) before queueing.
It's a little bit of a hack - edit the handler method as needed. I would modify the delayed_jobs table to have a class/object_id to make for cleaner code and more efficient table scans if your jobs table is large or if you do this with other types of jobs.
This question also looks relevant:
How to cancel scheduled job with delayed_job in Rails?